Cool new planet money episode! https://www.npr.org/2019/08/09/749938354/episode-932-deep-learning-with-the-elephants
*Thread Reply:* Here's AI for Earth's article about the project: https://news.microsoft.com/on-the-issues/2018/08/09/can-sound-help-save-a-dwindling-elephant-population-scientists-using-ai-think-so/
*Thread Reply:* great links. listening to the episode now. on the topic of elephants, here is an article about a project we worked on with vulcan, which uses cv to count elephants, if anyone is interested.
https://engineering.vulcan.com/2018_1112_How-many-elephants-are-there.aspx
*Thread Reply:* moved to here: http://tree.westus.cloudapp.azure.com/shiny/
Intros are a great idea Ben! I'm Sara, I work on species recognition in camera traps, trying to figure out how to train models that generalize well and can handle rare or previously unseen species. I published the Caltech Camera Traps dataset (https://beerys.github.io/CaltechCameraTraps/), run the iWildCam competition for FGVC at CVPR (https://www.kaggle.com/c/iwildcam-2019-fgvc6), trained the Megadetector for Microsoft AI for Earth (https://github.com/microsoft/CameraTraps), and am currently interning at Google working with Wildlife Insights(https://wildlifeinsights.org/home-0). Really happy to see so many interested people on this slack!
Hi guys! I’m Siyu and I work at the AI for Earth program at Microsoft. I’ve been mostly working on camera traps too: curating datasets from partners (the public ones are on http://lila.science), adding training data to and operationalizing the Megadetector that Sara trained, and making batch inference using the Megadetector available to organizations through an API (https://github.com/microsoft/CameraTraps/tree/master/api/batch_processing). I’ve also done some projects using satellite images, and am learning to use Pangeo. I’m a part of a Sustainability Garage at Microsoft and we have hackathon projects with NGOs such as The Ocean Cleanup (detecting plastic debris in rivers).
Hello! My name is Lily and I’m a PhD student at Harvard working on applications of AI and ML to wildlife conservation, specifically combating illegal wildlife poaching through more intelligent ranger patrols. Here’s a recent video highlighting our project: https://www.youtube.com/watch?v=85bRbCcwiNg
hi everyone -- great intros; such an interesting group! i am jon van oast and i am a developer at wild me, a 501(c)3 non-profit in the u.s. which maintains open source software for wildlife conservation. our work centers around wildbook, which is a platform for collecting and analyzing wildlife data, based mostly on photo ID of individual animals. we use ai/ml in terms of both computer vision (detection, identification, etc), as well as some other uses, like machine translation and analysis of social media (to look for wildlife photos). https://wildme.org | https://wildbook.org | https://github.com/WildbookOrg/Wildbook
*Thread Reply:* Thank you, @Sara Beery!
*Thread Reply:* Neat!! My friend just showed me the app Seek this weekend — very impressive vision work and so fun to use!
*Thread Reply:* Awesome! Thank you! Lots of room for making it better, but off to a reasonable start 😃
*Thread Reply:* just updating, due to a small bit of housecleaning, the demo now lives here: http://tree.westus.cloudapp.azure.com/shiny/
*Thread Reply:* Hey Liz, That was a nice talk I saw at KDD, sorry you weren’t there.
*Thread Reply:* Thank you, Ben! I’m also sorry I wasn’t there. I would have really liked to hear more about your work, but I hope to catch you at the next gathering!
*Thread Reply:* Jon, sounds great to me! Feel free to start one and post about it on the general channel so people can join 🙂
*Thread Reply:* Hey @Bourhan, Dan Stowell is a great contact in this space. I've worked on bat species detection and classification and audio in the past and would be happy to chat more. https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005995
Hello everyone!
We (@Stefan Schneider & @Sara Beery) are considering putting together a WACV workshop proposal focusing on Individual Re-ID in images, video, and acoustics. Individual Re-ID involves using a range of methods, from simple statistics, to advanced methods in machine learning, which are used to re-identify an animal individual previously recorded within a database. Here are three fantastic examples, covering the three data sources: Image: https://www.researchgate.net/publication/320609526_Wildbook_Crowdsourcing_computer_vision_and_data_science_for_conservation, Video: https://advances.sciencemag.org/content/5/9/eaaw0736, and Audio: https://royalsocietypublishing.org/doi/full/10.1098/rsif.2018.0940.
We want to get an idea for the community's interest, and get feedback on what would make this a valuable, worthwhile workshop. Please let us know if you have suggestions for speakers, topics, or good work in the field that we should try to include!
*Thread Reply:* thanks for posting! (and thanks for the link to our paper!) ... i will see if anyone from the wildbook team is going to be attending.
Hi everyone! I’m Riccardo. Just like @Sreejith Menon, I was Prof. Berger-Wolf’s student. My research work was about developing a sensor system to track wild baboons behavior. We build both the tracking collars and bracelet, and the machine learning framework to analyze the data and extract information about the behaviors. The research project was a collaboration among the University of Illinois at Chicago, UC Davis and Politecnico di Milano (Italy).
I recently co-founded Wepo Inc. (https://wepo.io), a software consulting company with a focus on AI and Machine Learning. We’re looking forward to continuing our projects and effort in the field of environment and conservation.
Really happy to join you and to be in this community!
*Thread Reply:* Welcome @Alexis Joly, i’m a big fan of the lifeCLEF work!
I spoke at the BiodiversityNext Conference this week (https://biodiversitynext.org/), along with @Dave Thau, Serge Belongie, and many others. There were 4 different Machine Learning sessions, all of which were so full they were standing room only, and I was impressed by the number of awesome ML applications and methods being considered. There were lots of image ID, audio ID, and NLP projects, lots of need for ML expertise, and there wasn't a lot of existing crossover with this community. I'm hoping to connect these communities moving forward!
Here are few cool demos that were released for the "Advancing biodiversity research through artificial intelligence" session:
You can try out the Microsoft AI for Earth MegaDetector on TFHub here: https://overlay.sandbox.google.com/embed?overlay_name=megadetector_v3
You can try the first published AI model from the partnership between GBIF and Visipedia here: https://overlay.corp.google.com/?overlay_name=mushroom_recognizer_v2 , this model was built from the svampeatlas (http://www.svampeatlas.dk/) Fungi dataset from last year's FGVCx Fungi challenge.
Wildlife Insights now has a public "Try AI" page where you can test out their model: https://www.wildlifeinsights.org/try-ai
*Thread Reply:* Very cool. Any pointers to the NLP projects?
*Thread Reply:* Most of the people I talked to were working on OCR/NLP for old museum specimens and documents. I'll see if I can find a contact.
*Thread Reply:* @Laurens Hogeweg or @Mike Trizna can you point to an OCR/NLP project for museum specimens? I know a few were mentioned.
*Thread Reply:* I think OCR for printed text is more or less solved. You could use object detection to find the labels in a specimen and then perform OCR.
*Thread Reply:* For handwriting I know only this one: https://www.universiteitleiden.nl/onderzoek/onderzoeksprojecten/wiskunde-en-natuurwetenschappen/making-sense-of-illustrated-handwritten-archives
*Thread Reply:* That's what I was thinking of! Thanks 🙂
*Thread Reply:* And please introduce yourselves!
Hi everyone, I am the Co-Founder of www.apic.ai We preserve biodiversity with the help of honey bees as biosensors by using some computer vision techniques to analyze the health of the bees in order to use them as a proxy for the health of all pollinators in the environment.
We are based in Karlsruhe, Germany, so if anyone is in the area feel free to drop by for a coffee. Furthermore, if any one of you is currently at ICCV too, I would love to grab some lunch or coffee with you. 🙂
More info about us: https://www.apic.ai , https://about.google/stories/save-the-bees/
I am excited to join this slack 🙂
*Thread Reply:* Hi Wilfried, here is our cs related publication: http://openaccess.thecvf.com/content_ICCVW_2019/papers/CVWC/Marstaller_DeepBees_-_Building_and_Scaling_Convolutional_Neuronal_Nets_For_Fast_ICCVW_2019_paper.pdf
We are using SSD for the pollen detection, but plaing to switch to tinyYOLO in the future, when DeepBees goes on device.
For BeeDetection in general, we use something similar, to what K. Bozek, L. Hebert, A. S. Mikheyev, and G. J. Stephens. Towards dense object tracking in a 2d honeybee hive. used. But everything should be in the paper 😉
Is their a particular reason why you are interested?
*Thread Reply:* Hi Frederic, cool - thank you! I used to be a data scientist and work as a PHd student for 1,5 years with Harald Meimberg (Univ. of natural ressources & life sciences) in Vienna. We had similar ideas. I am really interested in models behind solutions but working with completely different models (GPLVMs) for image analysis.
*Thread Reply:* That sounds great. If you think he is still interested to collaborate on this topic, it would be awesome if you can connect me with Harald Meimberg.
What kind of ideas did you have?
*Thread Reply:* Sorry for my delay (I was at a conference). I am working on novel models for object classification using nearly 0 parameters (e.g.: layers or number of neurons), which are extracting separated features first and do the classification afterwards to decrease the number of needed training exampels
I will chat with Harald Meimber aboud this topic
*Thread Reply:* I don't know much about it, but there's been some work done on passive acoustic monitoring of whales that deals specifically with underwater sound! Here's an example of a recent paper: https://besjournals.onlinelibrary.wiley.com/doi/10.1111/2041-210X.13244
*Thread Reply:* @Tanya Birch could you connect Silvia to the people in Geo working on whale audio? It seems like a good place to start 🙂
*Thread Reply:* Someone on my team worked with NOAA Fisheries to make classifiers to help processing their underwater recordings of beluga whale calls. I imagine these are quite different from fish voice, but if you’d like, I can introduce you to the NOAA people who ran the project? Feel free to email me (yasiyu@microsoft.com) with a brief description of your project goals.
*Thread Reply:* Hi Silvia, We (Max Planck Institute of Animal Behavior) have some people in our lab with good idea of recording underwater data, we also have some data on fishes that communicate underwater (e.g. Danionella). Please email me (hnaik@ab.mpg.de) and I can put you in touch with some biologists in our lab.
*Thread Reply:* @Silvia Zuffi there is a group on this topic over on wildlabs. might be worth checking out? (and there are probably other threads on there too)
https://www.wildlabs.net/community/group/acoustic-monitoring
*Thread Reply:* (e.g. search hydrophones perhaps?)
Hey everyone... this crowd is likely interested in a machine learning competition we just launched around Snapshot Serengeti data. Our hypothesis is that because we've done a bunch of infrastructure work to make sure competitor code runs on unseen test data, we will get models that are more generalizable to new data. We look forward to your submissions! https://aiforearth.drivendata.org/
*Thread Reply:* @Dan Morris, how does this relate to what @Sara Beery ran last year? Similar topic, different dataset? Be interesting to cross train from there.
*Thread Reply:* Related in the sense that it's all "ML for camera traps", different in (a) task (Sara's iWildCam competition focused on domain transfer, which is hard and necessarily requires some coarsening of the categories), (b) data set, and (c) competition structure (this one has competitors submitting code rather than results).
*Thread Reply:* In Aspen, Colorado as part of WACV2020: https://wacv20.wacv.net/
*Thread Reply:* See below, we're opening it up to remote submissions :)
There is a new tenure track professor position opening in machine learning in the Center for the Advanced Study of Collective Behaviour at the University of Konstanz! https://academicpositions.de/ad/university-of-konstanz/2020/tenure-track-professorship-of-machine-learning/139409
*Thread Reply:* Thanks for sharing! If anyone has any questions about the position, department, or Konstanz in general feel free reach out to me or @Hemal Naik
*Thread Reply:* This is awesome! Congrats on the funding 🙂
*Thread Reply:* Do you have the link to the schedule?
*Thread Reply:* Or do you need to sign up for the webex to see it?
*Thread Reply:* https://docs.google.com/document/d/e/2PACX-1vQ5j59KWQCI7zvH-scVx6dh58ceC-ZiHJKRFl2Y64vC7rFWeJybivZBde3EpAxwGSUmmKY2THUjXGES/pub @Hannah Kerner add my date when you have a sec?
Here is a link to Tony’s excellent paper. https://www.mdpi.com/2072-4292/11/7/768
Posting again for the meetup on machine learning for environmental remote sensing (thanks to @Hannah Kerner), this friday at 11am PT will by Dr. Sherrie Wang on weak supervision in remote sensing. Check out her great paper here: https://www.mdpi.com/2072-4292/12/2/207 at the Webex link: https://umd.webex.com/meet/hkerner
*Thread Reply:* Is there a website or mailing list we can join/share to spread the word about these seminars?
*Thread Reply:* ML for Remote Sensing Reading & Discussion Group (@Hannah Kerner) I think needs to add people individually?
*Thread Reply:* were you able to join? I wasn’t sure what the settings were.
*Thread Reply:* I was able to submit a form to apply. I mainly want to share this with some folks at our lab working on ML and remote sensing who aren't part of this Slack group!
*Thread Reply:* You should be able to request to join the google group: https://groups.google.com/d/forum/ml4rs/join then I or another organizer will approve it
*Thread Reply:* let me know if you have any issues!
*Thread Reply:* @Amrita Gupta you're also totally welcome to invite people to this slack if you think they're interested!!
*Thread Reply:* I would definitely recommend using the raw images. Beyond just distortions, I would imagine especially at the edge of frames some birds are blurred out or disappear entirely in the tile making process. Especially if you can assume that the ground is approximately flat, which, for the everglades I guess is mostly true, and the drone is flying at a fixed height, I don’t think the georectification will give you much extra information. You should be able to correct for most of the double counting by taking fraction of image overlap into account without needing the orthomosaic. Of course neither approach accounts for moving birds. I’d be curious to hear more about the project!
*Thread Reply:* i’m thinking along the same lines. Eventually I think our partners will want the spatial location for other uses. Have you ever seen posthoc georectification? I don’t have a ton of experience here, but my hope is that we can get the camera information and warping from agisoft photoscan and apply those changes to annotations as well as the images. sound crazy?
*Thread Reply:* Thanks, @Dan Morris! Everyone, please let me know if you have any questions!
*Thread Reply:* adding that i’ve worked with Volker and he and Anna are incredibly kind and talented scientists. highly recommended.
*Thread Reply:* Hey Zac, that’s where we are too. https://www.mdpi.com/2072-4292/11/11/1309, also see all the great work by others on online learning and active learning with human in the loop
*Thread Reply:* https://link.springer.com/article/10.1007/s13218-020-00631-4
*Thread Reply:* also https://www.aaai.org/ojs/index.php/AAAI/article/view/4288
*Thread Reply:* Ok, awesome! Everyone else in the lab I work at works on medical images, so I’m familiar with the topics of active learning, semi-supervised learning, and self-supervised learning in that domain, but I’m still learning all what has been done so far for with wildlife image tasks. Thanks for all the links!
The third annual iWildCam camera trap competition just launched! https://www.kaggle.com/c/iwildcam-2020-fgvc7
This year we are focusing on multimodality: how to best combine multiple available data streams to improve conservation-focused ML results. For every camera trap in both train and test we provide paired multispectral remote sensing imagery from the same location that competitors can use to try to improve generalization to new cameras. Take a crack at it, and let me or @Elijah Cole (Deactivated) know if you have any questions!
If anyone hasn't signed up yet, you can still register for the WILDLABS virtual meetup on Acoustic Monitoring this afternoon! https://www.wildlabs.net/resources/community-announcements/wildlabs-virtual-meetup-invitation-acoustic-monitoring
From the talk today, source of open source recording data for australia https://acousticobservatory.org/
This Digital Data in Biodiversity conference is going fully digital this year, and registration fees are now optional:
Also, WILDLABS has brought back it's monthly digest on conservation tech. See the March write-up here:https://mailchi.mp/wildlabs/digest-march-2020?e=7b9d646075
*Thread Reply:* Please join us for the fifth CompSust Open Graduate Seminar for spring 2020! This talk will take place on Friday, April 03, 2020, from 1:30–2:30 pm Eastern Time.
Sara Beery from CalTech will be presenting on "Improving Computer Vision for Camera Traps: Leveraging Practitioner Insight to Build Solutions for Real-World Challenges". Details about the talk and how to join are provided below.
Please join the meeting via the link: tinyurl.com/cogs2020
Join by telephone
Join by SIP conference room system Meeting ID: 138 884 216 138884216@zoomcrc.com
Title: Improving Computer Vision for Camera Traps: Leveraging Practitioner Insight to Build Solutions for Real-World Challenges
Abstract: Camera traps are widely used to monitor animal populations and behavior, and generate vast amounts of data. There is a demonstrated need for machine learning models that can automate the process of detecting and classifying animals in camera trap images. Previous work has shown exciting results on automated species classification in camera trap data, but further analysis has shown that these results do not generalize to new cameras or new geographical regions, and struggle to categorize rare species or poor quality images. Consequently, very few organizations have successfully deployed machine learning tools for camera trap image review. I will discuss my recent work tackling these real-world challenges with improved model architectures, data de-siloing, and data augmentation methods, and building accessible tools for biologists with Microsoft AI for Earth and Wildlife Insights.
*Thread Reply:* My COGS talk starts in 10 minutes! I'm excited :)
*Thread Reply:* oh my... well, yes. a reminder perhaps we all should use passwords! 😮
*Thread Reply:* ( i had my audio on and was coding, kinda zoned out -- then came out of my code-fog to realize there was commotion.... haha.... sigh. )
*Thread Reply:* Yeah, I get the feeling the COGS organizers will definitely be adding a password after that 😕
*Thread Reply:* for sure. i have had 3 such meetings bombed in recent week+ ..... i guess all the middle schoolers left out of school with nothing to do. 😞 ha.... at least it is an excellent working example of how security by obscurity doesnt work well. 😄 excellent talk though. i have to try so hard not to get distracted by camera traps (until someone actually gives me official work on the subject i mean).
*Thread Reply:* Do it!!!! Camera traps for everyone!
*Thread Reply:* not until i get budget/instructions.
*Thread Reply:* we do have these real world giraffe camera trap images... and i am the giraffe tech lead.... mwahaha
*Thread Reply:* heck, i even want to start out with training a simple detector of camera-trap-source vs not-camera-trap-source. cuz who even knows what photos i have over there in giraffeland. #distracted
*Thread Reply:* I think it would probably be pretty easy to train, most camera trap images have those weird timestamps/logos.
*Thread Reply:* exactly. and that way we take the burden off the user from caring about that distinction. i agree/hope also that it would be a fairly easy task. maybe one of my "its a weekend in lockdown" unofficial dabblings....
I got this in a newsletter from AI LA. Not sure about if it’s going to be good or not, but I wanted to share if anyone is interested: https://www.eventbrite.com/e/computer-vision-on-the-edge-webinar-tickets-97793104809
*Thread Reply:* lemme try to rope @Jason Holmberg (Wild Me) and @Jason Parham in on this one!
*Thread Reply:* hi!! I know way too much about thiiiisss. This was most of my PhD. Here are the canonical citations. What would you like to know. https://www.annualreviews.org/doi/full/10.1146/annurev.ecolsys.110308.120159 https://onlinelibrary.wiley.com/doi/full/10.1111/j.2006.0906-7590.04596.x https://onlinelibrary.wiley.com/doi/full/10.1046/j.1466-822X.2003.00042.x
*Thread Reply:* @Ben Weinstein how about I read these and then we meet and discuss sometime next week?
*Thread Reply:* sure. Has to be between 1:30-3ish on a weekday to make sure the kid is asleep.
*Thread Reply:* Works for me! How about 1:30pm on Wednesday? I'll send a meeting invite 🙂
*Thread Reply:* one more for background https://onlinelibrary.wiley.com/doi/full/10.1111/j.1461-0248.2005.00792.x
*Thread Reply:* Invited @Elijah Cole (Deactivated) to join the party 🙂
*Thread Reply:* Also worth checking out, may help automate anything you are planning to do https://wallaceecomod.github.io/
*Thread Reply:* @Ben Weinstein you are magnificent
*Thread Reply:* lol, you literally couldn’t have picked something easier for me to do.
*Thread Reply:* I am highly interested in that too. So thank you very much for the literature tips!
*Thread Reply:* https://www.pnas.org/content/114/49/12976.short
Conservation X Labs is hosting a free "ideathon" this Saturday to workshop conservation technology ideas: https://conservationxlabs.com/ideathon
*Thread Reply:* FieldKit is doing an earth day giveaway: https://www.fieldkit.org/blog/fieldkit50-earth-day-giveaway/
*Thread Reply:* Meredith Palmer gave workshop on how people can contribute as citizen scientists during quarantine: https://scistarter.org/go-on-safari-with-citizen-science
*Thread Reply:* part of NASA's #EarthDayAtHome activities include a cute game that helps the Ames supercomputer learn coral classifications
*Thread Reply:* http://nemonet.info/
*Thread Reply:* We’re doing a bioblitz on iNaturalist for offices everywhere, to which I’ve made zero contributions so far 🤓 https://www.inaturalist.org/projects/microsoft-bioblitz
*Thread Reply:* Also I just started using Ecosia, a search engine that uses ads profit to plant trees! https://www.ecosia.org/ Search results come from Bing
*Thread Reply:* sorry - is a new site ... will update original.
*Thread Reply:* np. yeah that was kind of ridiculously unclear the way i stated it. thanks for the nudge.
*Thread Reply:* Not sure if you meant to spell need with three e's, but I vote to keep it. Emphasizes the neeed 😁
A new project has appeared that Wild Me needs help with!
}
*Thread Reply:* New members, intro yourselves! Where are you from? What are you working on? What problems are you interested in?
*Thread Reply:* Sure, I’ll introduce myself. 🙂 I’m a Swedish-speaking Finn who lives in Switzerland. Currently, I’m working on deep learning for computer graphics rendering at NVIDIA. It has nothing to do with conservation unfortunately, but I’m hoping to be inspired by others on this channel and maybe start some side projects.
*Thread Reply:* There is some cool work on graphics rendering for conservation coming from Michael Black's group, check out @Silvia Zuffi's ICCV 2019 paper https://arxiv.org/abs/1908.07201
*Thread Reply:* Ooh, thanks for sharing that. I’ll read it asap. 😄
*Thread Reply:* Hi! My name is Miguel and I'm currently working on rodent brain lesion segmentation in Finland, so basically segmentation on medical images with deep learning :) I find very interesting everything related to environment + computer science, and more specifically, deep learning, so I'm quite interested in following this topic more closely :) Nice to meet you and thanks for letting me in this channel.
*Thread Reply:* Hi Suzanne! I met Topher White at a talk last year; rfcx is awesome
*Thread Reply:* @Siyu Yang Amazing! Indeed it is 🙂
*Thread Reply:* Welcome. If you haven’t already been put in touch with @Siyu Yang, check out the megadetector for the camera trap needs. Just out curiosity, what are the challenges/workflow for computer vision and drug discovery?
*Thread Reply:* Hey Ben, I've been making great use of the megadetector (Thanks @Siyu Yang!)
Where I worked (recursionpharma.com) it was about analyzing cell morphology to characterize disease models and their interactions with potential drug candidates. It was high-throughput screening.
The biggest challenges are deconvolving experimental variation with actual biology a la https://www.kaggle.com/c/recursion-cellular-image-classification and just the general difficulty in associating morphological differences with underlying biology
*Thread Reply:* This paper could be interesting, it’s about combining object detection and instance re-identification from multiple views. The setup is to detect urban trees in multiple streetview images and combine these detections in order to create an inventory of geolocalized trees. http://openaccess.thecvf.com/content_ICCV_2019/papers/Nassar_Simultaneous_Multi-View_Instance_Detection_With_Learned_Geometric_Soft-Constraints_ICCV_2019_paper.pdf
*Thread Reply:* I'd look at the video object detection literature, previously there was a large focus on LSTM and/or 3D convolution-based approaches, recently it seems that attention (like my context r-cnn paper) is very popular and works quite well.
*Thread Reply:* bald eagle population rebounds are pretty spectacular
*Thread Reply:* Northern elephant seals for sure!
*Thread Reply:* antarctic whale recovery rates are pretty stellar, mostly humpbacks. If current estimates of 1900s era harvesting rates are true, its amazing there are any whales in the southern ocean.
*Thread Reply:* Follow-up question, what about these conservation efforts made them so effective?
*Thread Reply:* there was a specific and targeted set of threats that came from a narrow set of actors.
*Thread Reply:* eagles -> ddt , whales -> sailors. The diffuse nature of most conservation challenges (climate, habitat loss) prevents specific legislation or actors. Also, in both case, the demand side of the equation made sense. Whale oil =/meat became less desirable after petroleum deposits were found. There were alternatives to other pesticides besides DDT. There was a technical solution that more or less reduced the incentive to harvest. Which is why you still get pangolin harvests, even though there is a specific actor (poachers). The demand remains.
*Thread Reply:* Any success stories you can think of not centered around a single species? Like, I don't know, a regional success story?
*Thread Reply:* Migratory Bird Act which is currently being scrutinized.
*Thread Reply:* much more controversial -> tiger reserves in India. There are alot more tigers and its been a boon for wildlife, but there are alot of questions about human-wildlife conflict and fairness. Outside of my expertise.
*Thread Reply:* you can definitely find some indian conservationalists (my friend anusha shankar will talk to you forever) if you want more detail there.
*Thread Reply:* https://anushashankar.weebly.com/ (tell her I sent ya)
*Thread Reply:* it makes for good reading. https://www.nature.com/articles/d41586-019-03267-z
*Thread Reply:* For >1 species how about waterfowl conservation in North America? https://en.wikipedia.org/wiki/North_American_Waterfowl_Management_Plan (also wetlands conservation, public-private partnerships, etc.)
*Thread Reply:* Clean Water Act is a big one. Not just with conservation in mind, but hugely impactful.
*Thread Reply:* So, I'm seeing trends of stabilization/regrowth of specific populations - particularly when it can be made financially viable, and large-scale far-reaching protective legislation. What else is successful?
*Thread Reply:* Not a conservationist but I read Noah’s Choice last year which tells stories around the Endangered Species Act. Slightly dated but very good perspective on what makes conservation policies successful/challenging and a very fun read https://www.goodreads.com/book/show/24022445
*Thread Reply:* California Condor recovery program! https://www.fws.gov/cno/es/calcondor/CondorResources.cfm
*Thread Reply:* sounds interesting! to be sure i'm getting time zones correct: that's 12pm ET on wednesday?
*Thread Reply:* I'd love to hear your talk! How lost would I be with my "very good at ordering tacos" Spanish?
*Thread Reply:* You’ll be fine, I still haven’t heard back about what language I should be speaking in. So all the slides are in English right now. I might speak in Spanish, but it’s really up to the talk organizer.
*Thread Reply:* @Elizabeth Bondi is! So is @Benjamin Kellenberger!
*Thread Reply:* Hi @Mikey Tabak! My experience is primarily with thermal imagery from drones, and so far we've found that in this context, Faster RCNN with VGG-16 works better compared with some other baselines. More detailed results can be found in our recent paper here: https://teamcore.seas.harvard.edu/files/teamcore/files/2020_07_teamcore_wacv_birdsai.pdf. Please let me know if you have any questions!
*Thread Reply:* @Mikey Tabak can you show a photo to get some details? Lots/few of objects in an image? Here is a recent nesting bird drone model I did. Transfer learned from my tree model available here (https://deepforest.readthedocs.io/) Keras-retinanet is the backbone engine.
*Thread Reply:* Good call @Ben Weinstein @Elizabeth Bondi! The success of aerial wildlife detection depends on the background complexity and the number of annotations available. For example, I’ve had good results with rarely occurring animals, but it required some tweaking. Here’s our work on weakly-supervised mammal detection from drones (done with a modified ResNet-18): http://openaccess.thecvf.com/content_CVPRW_2019/papers/EarthVision/Kellenberger_When_a_Few_Clicks_Make_All_the_Difference_Improving_Weakly-Supervised_CVPRW_2019_paper.pdf On the other hand, I also recently tried extremely densely populated scenes. I was able to use the same model, but had to use some other tricks (paper in the making). I guess we really need some examples to get a better idea.
In the meantime, my recommendations are: • Think first if you really need bounding boxes, or if points suffice. In the latter case, you can use smaller models (as I did), which are easier to implement, better to train on small datasets, and faster. • If you need bounding boxes, both Ben and Liz’ recommendations (Faster R-CNN and RetinaNet) are great. • As an underlying feature extractor (part of any convolutional neural network), I personally like ResNet. VGG as mentioned by Liz also is extremely powerful, but super big. One last question: do you already have labels? If you need to annotate images, I’ll cheekily take this opportunity to advertise our project as well. It’s called “AIDE” and is an annotation interface with deep learning models built-in (such as RetinaNet): https://github.com/microsoft/aerial_wildlife_detection (I’ll leave a proper introduction of AIDE in the chat for another time 🙂)
*Thread Reply:* Thank you all for your ideas! Unfortunately I can't share any images because of intellectual property issues. We were debating having technicians "paint" the objects of interest vs bounding boxes. I have not seen anyone use points instead of boxes as you mentioned @Benjamin Kellenberger. This might be the most sensible for our dataset.
*Thread Reply:* A followup question: What types of gui software do y'all use to have technicians provide bounding boxes or points of objects for detection models?
*Thread Reply:* https://rectlabel.com/
*Thread Reply:* We have our own software AIDE that supports labels, points, bounding boxes, and segmentation masks and has deep learning models built-in: https://github.com/microsoft/aerial_wildlife_detection
AIDE is under active development (by me), but already quite usable (and being used for wildlife conservation at this moment). Plus, it’s completely free and open source.
*Thread Reply:* Hi there, we recently finished a project for segmenting invasive plants from UAV RGB images with around 3.8 mm ground-sampling distance. A student of mine did a comprehensive study also on different networks. We didn't work with detection networks though. Still, let me know if you have any other questions. Maybe I can share some insights. We're also looking to publish this work in the next time. Code should be available soon as well!
*Thread Reply:* Thank you @Benjamin Kellenberger and @Lukas Liebel!
*Thread Reply:* Another followup question on analyzing drone images: What type of packages/code have y'all used to take aerial photos that are georeferenced, crop them, detect objects (or segment pixels), and then put them back into a georeferenced format. This seems like a pretty big challenge that I assume others have addressed.
*Thread Reply:* You mean, something like 'rasterio' to read and write georeferenced images with python?
*Thread Reply:* I’ve written a few gists around this idea, you’ll need to be a bit more specific, but this will give you the flavor https://gist.github.com/bw4sz/e2fff9c9df0ae26bd2bfa8953ec4a24c
*Thread Reply:* As far as cropping patches for training from a bigger image or mosaic is concerned, windowed reading/writing using rasterio is very helpful. It also allows you to easily save prediction results/labels/whatever (basically everything that can be converted to a numpy array and, in the easiest case, is of the same shape as your image data) to georeferenced tiffs. A simple workflow could be: • open image as dataset • read metadata (shape, channels, georef) • read a patch from a specified position in the image (saves you from reading an enormous image to your poor RAM) • predict that patch • repeat these two steps until you accumulated all of your desired predictions in a numpy array (same size as the original image given by the image's metadata) • open a new dataset to output the predictions (e.g., a new GeoTIFF file) • copy the metadata from the image to this new dataset (you'll probably want to adjust the number of bands/channels) including the georeference • write the numpy array to this new dataset • done. You successfully generated a fully georeferenced prediction for your image. The only thing left to do is to develop your magic black box predictor 😛
*Thread Reply:* I just wrote some of this logic for wrapping into tensorflow records for nice fast training. beware large sizes on HDD. Much as @Lukas Liebel alludes to https://github.com/weecology/DeepTreeAttention/blob/6361e4c5eda907d83643cfd789c0bef78be3fe2a/DeepTreeAttention/generators/make_dataset.py#L53
*Thread Reply:* Thank you both for your responses! I was hoping to use rasterio (@Ben Weinstein it looks like you're using this too) but I haven't been able to get it installed. I'm (unfortunately) using Windows now and the installation is problematic. I'm hoping they'll respond to my question about it on their github page. My general plan is to get large aerial image files, split them up while preserving the georeferencing into chips, have technicians paint masks (for object segmentation) on some of the images, train a model on these annotated images, use the model on non-annotated images, and reassemble the chips into one georeferenced file, so I can have the locations of the objects in the images.
*Thread Reply:* And @Lukas Liebel, yes I'll be doing this in Python
*Thread Reply:* @Mikey Tabak Well, your plan definitely sounds doable 🙂 In general, GDAL might also help you (either the standalone command line tool or the python package). Especially when splitting the training images into tiles it may be more convenient than re-implementing your own routine in python. Both ways should work with minimal effort though.
*Thread Reply:* @Mikey Tabak Regarding the installation of rasterio: In my experience (on Linux machines) it was always just a matter of a simple 'pip install rasterio' call. I'd suggest to use Anaconda. Chances are rasterio is also available as a conda package and Anaconda also comes with pip on Windows I guess. Another workaround could be using a Docker container. Even though this is kind of an extra mile to go, you may want to use it for training or other future tasks at some point anyway. I can help you with a minimal setup if you want to try that. It's dead simple for most of our daily applications but requires quite some Googling if you're not familiar with the terms and typical problems.
*Thread Reply:* Thanks @Lukas Liebel I'm still working on getting rasterio installed and nothing is working. I might use R instead for this task. It interfaces really well gdal.
*Thread Reply:* @Benjamin Kellenberger Do you have an estimate of when your AIDE software will support semantic segmentation? I don't need to use the software to run the AI model, I just want to be able to have technicians "paint" the images and then be able to access the annotations. The software is really impressive for its other functions!
*Thread Reply:* @Mikey Tabak Thanks a lot for considering our software! 🙂 AIDE in its current version (v2) fully supports semantic segmentation and even has U-Net built-in! Annotation is still a bit manual (without tools like Photoshop’s magic wand built-in), but we’ll be adding those as we move along. You can find a demo instance of it (v1) here. Feel free to let me know if you have more questions, wishes for functionality, or if you find errors.
*Thread Reply:* Thanks @Benjamin Kellenberger! I got segmentation running on my project. I wasn't configuring it properly and that's why I didn't see the interface for the segmentation. This is the perfect tool for what I'm trying to do. I have some little questions about it, but I'm going to put these on the github page as others will likely have similar questions.
To keep the brainstorming + collaboration going (i’m really glad to see that this channel has gotten so active, thanks @Sara Beery and A14Earth for $). If anyone out there is wanting to work with me on hyperspectral image classification. I am creating an open-source implementation of this great paper (Hang et al. 2020 “Hyperspectral Image Classification with Attention Aided CNNs”) here: https://github.com/Weecology/DeepTreeAttention. Only 1 week in. All are welcome. I’m going to recreate their results using open source benchmarks.
*Thread Reply:* here is the model architecture
Our Kaggle competition is live. If you have any questions about the competition, please let me know! https://www.kaggle.com/c/birdsong-recognition/overview
*Thread Reply:* If you have questions, Questions concerning
ROSES-20 Amendment 32: A.41 Citizen Science for Earth Systems Program Final Text and Due Dates Released
The primary goal of the
ROSES-2020 Amendment 32 releases final text and due dates for
On or about June 12, 2020, this Amendment to the NASA Research Announcement "Research Opportunities in Space and Earth Sciences (ROSES) 2020" (NNH20ZDA001N) will be posted on the NASA research opportunity homepage at http://solicitation.nasaprs.com/ROSES2020 and will appear on SARA's ROSES blog at: https://science.nasa.gov/researchers/sara/grant-solicitations/roses-2020/.
XPRIZE has a $10mil rainforest competition going on: https://www.xprize.org/prizes/rainforest > Rainforests cover less than 10% of the earth's land surface, but they house approximately 50 million inhabitants and over 50% of the planet's biodiversity. Although they are the most biodiverse ecosystems, there is a limited knowledge of everything that lives in these iconic environments. The value of the standing trees are not fully understood and our ability to assess this value is restricted because the rainforest environment is dense, vast, and complex. > > The winning team will develop novel technologies to rapidly and comprehensively survey rainforest biodiversity and use that data to improve our understanding of this complex ecosystem.
Hi all, we are looking for a postdoc in the NASA Harvest group at UMD College Park. Research topic is ML and EO applications for cropland & crop production forecasting in smallholder agriculture (field to national scales) focusing on Sub-saharan Africa. Please apply or share with potential applicants! Thank you! https://ejobs.umd.edu/postings/78351
*Thread Reply:* @Caleb Robinson, @Elijah Cole (Deactivated) you guys might know someone?
*Thread Reply:* Hi @Hannah Kerner! I don't know of anyone with an EO background looking for postdoc unfortunately
*Thread Reply:* I do know someone looking to apply to a CS PhD program
*Thread Reply:* I'll check back with them in 4-6 years 👍
*Thread Reply:* Candidates can also have an ML background but no EO background yet, as long as that's a direction they're interested in!
*Thread Reply:* @Hannah Kerner this just popped up on my scholar feed if useful (https://www.frontiersin.org/articles/10.3389/fenvs.2020.00085/full)
*Thread Reply:* thanks for sharing, definitely relevant!
I'm giving a WILDLABS "Tech Tutorial" this morning (8am PST) on getting started using ML for camera traps. It's geared more towards an ecology/non-ML audience so I didn't think to post it here, but if anyone is interested you may still be able to register! https://www.eventbrite.co.uk/e/tech-tutors-how-do-i-get-started-using-ml-for-my-camera-traps-tickets-109658752280?ref=estw
If you can't register, the recording will be posted on YouTube tomorrow :)
Context R-CNN is on the Google AI blog today! https://ai.googleblog.com/2020/06/leveraging-temporal-context-for-object.html
*Thread Reply:* Hi @Drew Gray, perhaps you know them already but https://www.nature.com/articles/srep23166 the scripps, san diego groups did alot of work here in the last decade, it would be good to hear from them.
*Thread Reply:* @Sam Kelly I talked to Bob earlier, and he's interested in edge-based solutions. You two should talk!
*Thread Reply:* I am. I emailed @Sara Beery about porting Megadetector to TFLite for embedded SOCs -- something that appears a overly ambitious for the current state of the art -- still, we have what's left of Moore's Law 🙂 Another area I've thought about is whether we can improve the PIR sensor accuracy using AI -- perhaps leveraging new Si aimed at "always on " audio processing. Very interested to hear what you are up to.
*Thread Reply:* I think lightweight detection is totally possible and even useful! It's just not as accurate, which is why we kept the MegaDetector a larger model.
*Thread Reply:* Oops -- I misunderstood. I need to understand accuracy tradeoffs more. In app space I have in mind, camera location independence is key. In any case, I should start small to limit the amount of trouble I get into
*Thread Reply:* I think the PIR sensor accuracy seems really interesting too!
*Thread Reply:* Hi @Bob Zak! Would love to chat to you about this! We have been thinking about some of this for a while at Conservation X Labs! Definitely some things I would love to discuss with you!
*Thread Reply:* @Sam Kelly Great! I just updated my profile with a skype address; (updated) phone in profile also works. I'm around now through the afternoon.
*Thread Reply:* I love this idea and it’s something I’ve been thinking about too. Maybe you could look into using NVIDIA’s Jetson Nano or Xavier NX as well.
*Thread Reply:* Yeah -- there seem like there might be a lot of hardware options. But (as a newb to the details) I have basic questions about MD model size; extent that it's been optimized to reduce said size; tolerance to thinks like int8 mapping; etc. I guess another option would be to start with same training set, but target a smaller (more edge friendly) model to start with. So many questions...
*Thread Reply:* lots of awesome innovation happening in this space - On the hardware front: https://github.com/basicmi/AI-Chip We are attempting what you are talking about with a YOLO-tiny / Darknet (or similar) base model with limited classes and transfer learning approach with pruning/quantization included. Will be sure to share how it goes
*Thread Reply:* So someone has already concluded that running current Inception-Resnetv2 MD (tuned, optimized, int8, etc.) doesn't work on one of these edge devices?
*Thread Reply:* @Bob Zak and @Sam Kelly I am also interested in intelligent camera traps. There are SoCs that can run high accuracy models and those the can run really low power. The trick is to find the middle ground that makes sense. You can run Inception-Resnetv2 at the edge, but the power might be more than you bargin for. But that also depends on how frequently you run a full inference.
*Thread Reply:* On further investigation, and after conversation with @Sam Kelly, I'm convinced that the "intelligent camera" needs to start somewhere other than Megadetector. MD is solving a much harder problem -- locating any animal in any arbitrary trail camera photo, with no a priori context. Vs. at the edge, you have all sorts of context -- e.g. you know what the (animal-less) scene is, under any lighting condition. Intuitively, it seems like this should really simplify the detection problem. Which I think is critical -- even with low power hardware, my sense is that deep networks like MD are orders of magnitude more energy intensive than can be supported on batteries. Even worse, the most interesting edge camera application (I think) is likely video, and here you need to keep up with frame rate.
*Thread Reply:* I agree MD, as it is, is too heavy for frame by frame detection at the edge. I would think you would have a cascade of models, starting with "something is out there", perhaps using motion detector or "smart PIR" as you mentioned in another thread. The fixed context of the camera, as you mentioned, should help with verifying there is an object of interest without having to run a complex model. Once you have a region of interest, you can run though a further cascade of models as needed:
- You may want to classify at a coarse level, similar to MD: animal/person/vehicle. This would be beneficial if the MD-"lite" model is lighter weight than the species classifier, or if you want fast alerts for human/vehicle hits (possible poachers, etc.).
- If you want species level classification, you can have models trained on the "common" species for the camera area.
- If you want individual classification (like we do for the BearID Project) you only need to run when there's a species match. You shouldn't need to classify every frame at each level. You can use some computer vision based object tracking algorithms (again the known background should make this easier) to follow the object through frames and only run the classifier on a few frames to make sure there is agreement. You could also run some of the models with lowered resolution to save cycles/energy. Also, not everything has to run "live".
You probably don't need the bits that say something interesting is there to run on every fame. Maybe you run a few at the beginning to make sure we want to record this, then rely on the object tracking to keep the keep recording until the object leaves the scene. You can run some of the other models in the background from the recording when nothing else is happening.
Any thoughts on a data backchannel such as LoRA or DASH7?
*Thread Reply:* A hierarchy of models definitely makes sense (although, depending on the size of each of the models, the energy cost of swapping the models around could be significant). In addition to the usual ai worries (training, accuracy, size, etc), I think you also have to worry about “value proposition” for the usage model relative to other solutions, especially considering energy costs. E.g., storage capacity, and energy per image is relatively cheap — if you can afford to wait for results, then doing all the processing offline makes a lot of sense. If you need the data urgently (e.g. for poaching detection, or “real-time” animal localization) then you have to consider the the alternative of sending thumbnail images via radio for backend processing — or, more generally, which models in the hierarchy to run locally, vs which to run in the backend. Two overall pieces of scaffolding would be good to have: an “energy budget” model (positing energy costs for acquisition, storage, processing, and wireless rx/tx). The other would be a list of usage cases and their key figures of merit. I’ve not looked at Lora or dash7, though I did poke around at some of the satellite startups. I also looked at rficient (https://www.iis.fraunhofer.de/content/dam/iis/en/doc/il/ics/ic-design/Datenblaetter/FactsheetWakeUpv4.pdf) as a potential technology for remote sensors (promising, still waiting on sample availability)
*Thread Reply:* @Bob Zak after our discussion, we actually tried to compress MD for on-device deployment, and unfortunately it is not technically possible with the current available libraries (some of the CNN operations are not available for tensorflow lite). The Yolo algorithms and EfficientNet seem to be the go-tos in this area. On the comms/backhaul - would be very interested to hear about this, and the animal tracking/tag world may be a good analogy - LTE/LoRa/Iridium/ARGOS/VHF. I know @Alasdair Davies has a module for Argos that I'm interested in. @Ed Miller 100% agree with the cascade idea and keeping algorithms tailored to predicted (or study related) classes. I would be very interested to hear experts thoughts on MD-lite (animal/no animal) vs robust study-specific species detectors.
*Thread Reply:* @Bob Zak & @Sam Kelly @Ed Miller - super interesting thread here. I think we are all in the same realm of thought regarding what energy is used in the field on the edge, suitable models and realistic constraints. Backhaul to send data (photos or otherwise) to the cloud for MD continues to interest me. I have done it over Iridium rudics (slowly) with thumbnails and ZSL are working on Instant Detect 2 to advance this (gateway has the modem, lower power sensors send to the gateway for processing in the field). Currently looking at an EfficentDet model for edge thermal image detection with Leptons for an initial detection to wake something beefier (still edge) only if it's "worth" waking in the camera space to get more confidence and to spend more power
*Thread Reply:* A lot to unpack, there. Has anyone looked at the impact of resolution and spectral content on animal target detection? E.g. MD is severely downsampled from the sensor data, but still ~300x300 rgb. At the other extreme, the PIR sensor is a effectively single pixel (maybe two, depending on how you think of it) in a single wavelength.
*Thread Reply:* https://openaccess.thecvf.com/content_cvpr_2017/papers/Huang_SpeedAccuracy_Trade-Offs_for_CVPR_2017_paper.pdf
Looks at (among many other salient options) the impact of image resolution -- testing 300x300 and 600x600. They find: > The effect of adjusting image size. It has been observed by other > authors that input resolution can significantly impact detection > accuracy. From our experiments, we observe that decreasing resolution > by a factor of two in both dimensions consistently lowers accuracy (by > 15.88% on average) but also reduces inference time by a relative > factor of 27.4% on average. One reason for this effect is that high > resolution inputs allow for small objects to be resolved. Figure 4(b), > which compares detector performance on large objects against that on > small objects, confirms that high resolution models lead to > significantly better mAP results on small objects (by a factor of 2 in > many cases) and somewhat better mAP results on large objects as > well. We also see that strong performance on small objects implies > strong performance on large objects in our models, (but not vice-versa > as SSD models do well on large objects but not small). I don't believe this paper explores the effect of narrowing the spectral input (e.g. of processing a black and white image). Is MD more accurate on daytime (color) photos, than on nightime, IR illuminated B&W photos?
*Thread Reply:* I've looked at the effect of performance vs input image resolution, and it doesn't change performance on big animals much but drastically effects performance on small animals or animals that are farther away (totally intuitive, you're shrinking an already-small object so it loses information). Also, MDv4 takes in much bigger than 300x300, it uses a keep-aspect-ratio resizer with a min dimension of 600
*Thread Reply:* performance on day vs night depends on species, there's a bigger effect on performance when you see a species outside of it's normal time (ie most raccoon images are taken at night, so it struggles with raccoons during the day)
*Thread Reply:* Anecdotally, I have seen similar performance even with downscaling with large animals (as noted by Sara). I would also point out that if the downscale catches the animal most of the time, then you could use it by default, any only run a larger image occasionally for sanity check (or if you really think something should be there, but the downscaled version didn't find it).
*Thread Reply:* To some extent the real question is what are you trying to do on the edge. If you're trying to detect elephants it's a very different question then trying to detect rodents using an off-the-shelf camera with a PIR sensor optimized for large mammals. I think most small mammals don't trigger the camera at all, and then if they do they're harder to detect using CV because they're small and camouflaged. I think edge systems will need to be open-source and adaptable so that the elephant/bear/big cat/large mammal folks and the rodent/bird/herp folks can both optimize them for their use cases.
*Thread Reply:* (Back to B&W performance) I guess the real test would be train a single channel version of MD on a dataset of photos converted to B&W. I wouldn't expect it to improve accuracy 🙂, but I bet it would be a net win "accuracy per Joule". Similar for reducing the precision of each channel. Or, more generally -- within a fixed computation budget, what is most efficient allocation of resources towards: model depth; number (and peak special sensitivity) of color channels; and precision/encoding of each channel? (I'll look, but if someone already knows) Are there references that cover this topic?
*Thread Reply:* Did someone mention elephants 🐘 @Sara Beery 😄
*Thread Reply:* Increasing image resolution helps classification performance on the iNaturalist 2017 dataset: https://arxiv.org/pdf/1806.06193.pdf (Table 3. this is not a detection task)
*Thread Reply:* @Bob Zak Maybe I'm not fully understanding the B&W model you suggest, but isn't there a danger that any savings from reducing from a 3 channel to 1 channel input will be dwarfed by all the later computation which will not be impacted by changing the input dimensionality.
*Thread Reply:* Yeah -- good point. My ignorance of model structure and resource allocation showing 😞 Certainly, the deeper the model, the less important the size of the input layer. Thanks for pointer to iNat paper!
*Thread Reply:* @Oisin Mac Aodha Thanks for the pointer to the FGVC paper. The paper is focused on the classification step, and only considered resolutions from 299x299 (89k pixels) to 560x560 (314k pixels). The 4x image size difference will have some compute/energy impact, but it does also provide a 15% improvement on the Top-1 error (29.93% -> 25.37%). This could be very useful for the BearID Project face classifier, as we currently only use a 150x150 pixel face image.
Some of the discussion earlier on this thread was referring to the image resolution for an Object Detection network. For example the BearID Project's bear face detector. We have tried this on input images from 15M pixels down to 307k pixels. My completely unquantified observation shows a huge reduction in compute with little effect on accuracy (at least for my single class detector). Perhaps we could run some sweeps to quantify this.
@Sara Beery and @Siyu Yang: has anyone run compute vs accuracy comparisons for Mega Detector?
Has anyone run such comparisons for other object detectors with more classes (like those trained on COCO, etc.)?
*Thread Reply:* Maybe one pointer would be the EfficientDet paper which explores models with different input resolution (but note that the model size is changing too). https://arxiv.org/abs/1911.09070
New job posting for a data scientist at a startup working on carbon removal & climate solutions: https://apply.workable.com/carbonplan/j/E6E55611B9/
WILDLABS is running their 3rd annual Conservation Tech Survey! Let's make sure that CVML researchers have a voice in the survey: https://mailchi.mp/wildlabs/community-survey-2020?e=7b9d646075
Hi all 👋, glad to be joining. At Appsilon we have an AI for Good initiative, which led me into the conservation space. Curious to see what you are up to and will be glad to share our results here.
*Thread Reply:* Welcome! Loved reading the blogs about your work with camera traps in Gabon. Excited to see what comes it. Have you worked at all with passive acoustic devices/data? I am currently trying to figure out the best way to detect lemur calls in my continuous audio recordings from Madagascar.
*Thread Reply:* Hi Carly, I don't have much experience with the analysis of acoustic data, but would be interested to read up on it. Can you recommend papers/blogs?
*Thread Reply:* I can definitely go down the rabbit hole with that, but let me pull together some of the reviews and notable case studies, at least for terrestrial use. The marine mammal community uses acoustic monitoring via hydrophones extensively, but I am more familiar with the literature for terrestrial animals like my study species
*Thread Reply:* Marques et al (2013)-Estimating animal population density using passive acoustics. Heinicke et al (2015)-Assessing the performance of a semi-automated acoustic monitoring system for primates. Sánchez-Gendriz et al (2017)-A methodology for analyzing biological choruses from long-term passive acoustic monitoring in natural areas. Gibb et al (2018)-Emerging opportunities and challenges for passive acoustics in ecological assessment and monitoring. Kershenbaum et al (2019)-Tracking cryptic animals using acoustic multilateration: A system for long-range wolf detection. Wood et al (2019): Detecting small changes in populations at landscape scales: A bioacoustic site-occupancy framework. Sethi et al (2020): Characterizing soundscapes across diverse ecosystems using a universal acoustic feature set. Kvsn et al (2020): Bioacoustics data analysis-A taxonomy, survey and open challenges
*Thread Reply:* Thanks @Carly Batist! That is a helpful compilation!
*Thread Reply:* Have you already selected your hardware platform?
*Thread Reply:* I'm sure there are many ways to get started, but I might recommend reaching out to a local conservation group and offer your help? Many small conservation orgs don't have the bandwidth or resources to work with cutting edge tech, so it could be a nice way to get your hands dirty on a real problem.
This seems pretty cool: https://www.kickstarter.com/projects/opencv/opencv-ai-kit
*Thread Reply:* Definitely tricky. One option (of many) would just be to add the category and fine-tune using your previous model as a starting point? It might converge faster.
*Thread Reply:* Yes…so I guess the trick would be getting the weights from the smaller model to load correctly.
*Thread Reply:* just load everything except the final layer(s) where you map to the categories?
*Thread Reply:* It’s been a while since I looked at the code for loading weights and I can’t remember how hard it would be to just load a subset of the layers, but I think that approach makes sense. Thanks.
*Thread Reply:* Usually you can exclude layers by name
*Thread Reply:* Are you referring to Pytorch, I hope? Also, different topic but since I have your attention for a moment, is the code for your Context R-CNN model online yet? I’m really interested in that — such a smart idea.
*Thread Reply:* I believe tf and pytorch allow for excluding layers while loading. Context R-CNN is online in the tensorflow object detection API (sorry, this one's not pytorch!) https://github.com/tensorflow/models/blob/master/research/object_detection/README.md#context-r-cnn
*Thread Reply:* I just recently investigated this. There are not many solutions, mainly because training a CNN with only few examples (there are not many once a new class needs to be added) does not really work. Suggestions like few-shot and meta learning don’t work either, as they rely on many new classes/tasks with few examples.
The most recent version of AIDE solves this as follows: every time a new class is discovered, the model adds new output neurons to the classification head (of RetinaNet, ResNet, etc.) and initializes their weights by copying over some other classes’ values. This way, although the initial performance on the new class is not great, it does not alter the accuracy of the other classes at all. Thanks to active learning, the accuracy will improve over time, though. I have ideas on how to improve this mode in future releases, but for the time being, the software does support adding new classes on-the-fly, at least for the built-in models.
*Thread Reply:* Thanks Beni. I’m not sure what you mean by “adding new output neurons,” but I can imagine two ways to add a new class: Method 1) Add a new linear layer; for example if the current output layer dimensions are (bs x c) where bs is batch size and c is number of classes, then you add a new linear layer sized (c x c+1) on top of it, making your final output (bs x c+1). That means your model grows each time you add new classes. Method 2, which I think is what Sara was suggesting, is to replace the final layer (bs x c) with a new layer (bs x c+1) and simply re-learn the weights of that final layer. Out of curiosity, am I correct that you are using Method 1 in AIDE? To me, Method 2 seems a little easier to think about, but I can’t think of any major reason to prefer one over the other.
*Thread Reply:* Hi John, What I meant is the following: the final layer of a CNN maps from D to C, where D is the number of features of the penultimate layer (e.g., 512 for ResNet-18), and C is the number of classes. Effectively, this last layer contains weights of size DxC. Whenever there is a new class present, the models in AIDE expand this matrix and append a Dx1 vector (for 1 class) to it, which is initialized from a combination of existing weights. Effectively, there is no layer replacement or additional layer to add, but instead the final classification layer gets expanded to Dx(C+1). The advantage is that the weights for the existing classes can be retained, and the new class added without it confusing the model. I hope this helps!
*Thread Reply:* OK thanks Beni; yes that’s what I was referring to as Method 2 except that you are also transferring the old weights. Makes sense.
*Thread Reply:* Hey @Benjamin Kellenberger do you have any direct comparisons of this initialization vs random init when you increase the output layer? It makes total sense, and I'm curious if you get increased accuracy, increased training efficiency, or both? And when you do this do you retrain over your whole training set + the new examples? Do you freeze everything except the last layer or allow all weights to be updated?
*Thread Reply:* Hi @Sara Beery I have only run very quick tests, nothing substantial yet. I just trained a classification model (ResNet-18) on a remote sensing dataset with half of the classes (and images) removed. I then added the output neurons for a variable number of new classes and checked performance. Nothing was frozen; I always trained the full model end-to-end. With random initialization, the accuracy of the entire model went to zero; it basically put the whole model out of balance. Adding weights by copying other classes’ values resulted in still perfect accuracy for the trained classes, and (obviously) chance guess for the new class. I had to re-train on the new class for about the same number of epochs again as the initial model to get it up to speed.
I originally wanted to investigate this further and try out some more intelligent initialization schemes, but then dropped the idea. I might still do it just for AIDE.
*Thread Reply:* Gotcha. So to get good performance on the new classes you still need to train about the same amount of time over all the data. Thats kinda what I expected, but I agree it would be interesting to test further. Essentially this sounds ideal specifically for an active learning scenario? Where you'd still want the model to work "in the interim" while you're adding new training data or training on new data, before training is complete.
*Thread Reply:* Precisely. People have tried to address this with meta learning, attention models, etc., all of which sounds like an overkill or has hidden costs (meta learning requires many tasks with few examples). It would basically solve the one major problem I have with AIDE in that it allows getting models to provide somewhat decent predictions at the start instead of randomness.
*Thread Reply:* Makes total sense! Probably if you plan to retrain completely before eval though you can just take the simple path of increasing the final layer and randomly initializing all of it? If it gets to the same accuracy in the same amount of time. Just because it's easier to code up 🙂
*Thread Reply:* @Benjamin Kellenberger I would think that you could reduce your training time for new layers by progressively un-freezing the model. As you probably know, the standard approach in transfer learning is to start with all but the new top layer frozen (sometimes BatchNorm layers should also be unfrozen), and gradually unfreeze further and further down as the training progresses. The reason for that is that the lower-level layers in a CNN are doing simple things like identifying edges and curves, …and somewhat higher up, layers are identifying shapes, textures and so on, all of which are likely to still be useful for feeding into the topmost layers that are specific to your particular classification problem. But maybe I misunderstood when you said “Nothing was frozen; I always trained the full model end-to-end.”
*Thread Reply:* I've not really found it to make much difference in practice to unfreeze all right away or unfreeze them progressively.
*Thread Reply:* Same here to be honest; I found the learning signal from the few new examples to be so little that there was hardly any change to the early layers. Batch norm is another story… I had terrible experience with it and since then always replace it with instance norm. Yielded the same result and is much more stable.
*Thread Reply:* Interesting observations, thanks. I guess I had just accepted the orthodoxy without testing it.
*Thread Reply:* Welcome @John Brandt, @Hannah Kerner I think you and John have alot to discuss on agricultural landcover mapping in sub-Saharan africa, also @Tony Chang on multiple temporal fusion of satellite data for tree cover estimation.
*Thread Reply:* Yes! @John Brandt I read your paper recently about crop type classification with attention, would be great to talk more about that
*Thread Reply:* Hi @Hannah Kerner - Ben told me great things about your work. Nothing ever came of the attention paper for crop type classification, though I'd love to revisit it. I'm currently working on doing trees outside of forests monitoring with Sentinel. We're working in Ghana, Niger, Cameroon, and Malawi to develop jurisdictional maps on agroforestry and restoration implementation, and (separately) working with local governments across south and Central America to develop monitoring systems for restoration implementation. The preprint of the paper is currently updating to a new version on arxiv and hasn't been announced (since it had R&R), so I've attached it here. Would love to connect about your work and see if there is any overlap (since we're primarily focused on ag landscapes!)
*Thread Reply:* @Tony Chang similarly interested in chatting with you! We are doing multi temporal image fusion to help mitigate changes in growing season / leafing seasons so that we can use one global model, would love to chat
*Thread Reply:* Definitely John! Welcome to the group! Yeah, the global model challenge is one research that I'm exploring at the moment. Pretty interesting question regarding change detection too. Please feel free to connect with me. My email is tony@csp-inc.org
*Thread Reply:* This is really really interesting. My expectation would be that the virtue is similar to that derived from stochastic weight averaging (SWA) (https://arxiv.org/pdf/1803.05407.pdf) where the weights are frozen after epochs near the end of training, and the prediction is the average of the prediction of these late-stage epochs. The authors in the above article discuss in depth how averaging these good predictions results in an even better prediction, based on theoretical analyses of what the cross-entropy loss surface should look like. Based on this, starting with a summation of the two arms being already trained is somewhat similar to SWA where combining the two trained weights identifies a lower optima. The benefit of the paper you linked is that this lower optima is then the start of a new training cycle, where there are new patterns to learn between the overall network, that can then even find lower optima. I would expect that if you train the overall network end-to-end, you don't get the benefit of finding two optima, combining them to find a new even lower optima, and then having new spaces to explore
*Thread Reply:* By that logic I should (strongly) decrease the learning rate at the 2nd stage of learning to not hop out of any minimum I just found in the first stage of learning. I’m still playing with the right parameters for learning rate decay.
*Thread Reply:* if you use adabound you could probably just keep the learning rate as it would have converged to SGD by then
*Thread Reply:* https://arxiv.org/pdf/1902.09843.pdf
*Thread Reply:* no, i was using ADAM. looking now.
*Thread Reply:* basically just smoothly switches Adam to become SGD since SGD is known to generalize better than Adam but Adam is more stable early on
*Thread Reply:* Bias in the data we do have, natural world is long-tailed... both probably fall under "not enough data" one way or another
*Thread Reply:* • added. “Ecology is often focused on rare classes with large geographic variation.”
*Thread Reply:* In you data sharing section, it would be nice to talk about the opportunity to leverage bycatch when data is open
*Thread Reply:* ‘A role for museums as curators of digital information: many studies are focused on a single set of species, but record many incidental sightings that can be used as training data.’
*Thread Reply:* training data or just useful observations that have previously been left to rot on hard drives
*Thread Reply:* I would add something to acknowledge the diff toolkits ecologists and computer scientists are trained to use (and therefore stick with). e.g., every intro biostats class I've heard of uses R, as does most of the ecology world, whereas computer scientists breathe Python. Not saying one is better than the other, but it's hard to get either side to invest in learning a new programming language. If you wanted to really get at the core of the problem, you'd likely have to get undergrad programs to change the number or structure of required courses.
*Thread Reply:* with a nod to RopenSci, reticulate and https://keras.rstudio.com/
*Thread Reply:* And for data/model sharing, http://lila.science/ is a good example! Also Wildlife Insights, Xeno-Canto (which the warbleR package can pull directly from), MobySound/OrcaSound, the Macaulay library.
*Thread Reply:* shows how much outreach is needed, I’m a contributor to several of those things, but never heard of the warbleR package.
*Thread Reply:* Bioacousticians love that package. Also gibbonR, monitoR (all 3 build upon seewave and tuneR). And for web apps, ARBIMON/Sieve Analytics is good, but has a limit to storage/computing before you have to pay (they've also just merged with Rainforest Connection).
*Thread Reply:* From a behavioral ecology point of view, people were traditionally forced to choose between working in artificial lab environments to get good high resolution behavioral data, or working in natural environments and making do with much coarser behavioral information. Computer vision finally lets us get high quality behavioral datasets in truly natural environments.
*Thread Reply:* What is the next step for behavioral ecology in natural settings using these tools. What is the grand goal/obstacle?
*Thread Reply:* RE: cross-disciplinary collaboration difficulties. As a CS PhD student explicitly working in the "AI for sustainability" space for the past several years, I think it is still challenging to initiate collaborations between CS researchers and ecologists. Attending events like NACCB or US-IALE to learn about domain challenges can help, but is outside of typical venues for which AI researchers can get support to attend (in line with what Sara was saying about what is considered "CS research"). Even when we do get to participate in these events, I believe whether or not a collaboration comes out of it still comes down to knowing an "advocate" who will connect people on both sides.
*Thread Reply:* There is a nice debate here that parallels the discussion of double blind paper review versus active support for underrepresented groups that I think is really relevant to our community. https://www.nytimes.com/2020/07/16/arts/music/blind-auditions-orchestras-race.html?searchResultPosition=1
*Thread Reply:* Just a small note-I would use the term 'global south' rather than 'developing countries,' just to move away from the colonial terminology.
*Thread Reply:* And I'd add something about making actually equitable collaborations between researchers from the global south and north. Westerners going down and telling them what to do rather than listening and working with them also reinforces this neocolonial thinking. We need to be building in-situ capacity and long-term infrastructure. As it is now, someone gets a grant and goes down for 6 months to collect data and then leaves, which is 1) exploitative (often field assistants and local collaborators/org's are not included in pubs or even told the results) and 2) not sustainable.
*Thread Reply:* What are the measures that would help actualize this? I gave a talk virtually in Ecuador last week, and a lot of the discussion was focused on hardware resources, that just having access to the correct kinds of computational resources would be a really large step forward for a lot of field teams that I’ve been working with in Ecuador. I think there’s a trade off in terms of how much development should be focused on the cloud, which helps reduce the barrier to entry, since less coding is needed, versus getting data into the cloud, which can be really hard for my collaborators in Latin America.
*Thread Reply:* @Lily Xu any thoughts here from some of your recent organizing.
*Thread Reply:* Yes the hardware is definitely big, particularly because it facilitates in-country analyses rather than just data collection. But obsolescence is also an issue, especially with how fast tech is moving now. I work in Madagascar and the computers the research station got are really old/slow/broken/etc now and shipping is basically non-existent so it's a matter of someone flying new stuff in. I'd also say that whichever side of the trade-off you want, equitable partnerships are still integral and unfortunately in many places, not equitable at all.
*Thread Reply:* I by no means am trying to suggest I have answers to any of this! Just giving my experience being on the ecology side of things in the name of cross disciplinary collaboration! 🙂
*Thread Reply:* Hi Ben, just seeing this discussion now! I'm excited for your talk in September.
That's a great point connecting with the orchestras and faults of blind auditions/reviewing — many arguments for affirmative action apply here, too. I've done some reviewing for workshops recently (AI for Good and Mechanism Design for Social Good) which aren't blind, and during reviewer discussions I see people being a lot more mindful to authors and institutions from historically underrepresented areas, including Latin America.
In some ways, I think virtual conferences/events definitely help serve as an equalizer: conferences that formerly cost $2000+ to attend now cost $5–10. (Ariel Procaccia wrote an op-ed about the success of virtual EC)
I also think we need to, as Sara and others have mentioned here, reconsider what are "valuable" contributions to AI. For example, producing NLP datasets that are in languages other than English and other projects that help combat inequality throughout the AI pipeline, from data gathering to labeling to designing algorithms.
*Thread Reply:* Thanks @Bistra Dilkina for these thoughts, we often see this just from the ecology side. Also, i’m hoping to point to some of the park ranger optimization work under the heading of “Integrating measures of uncertainty in machine learning and ecology”.
*Thread Reply:* @Ben Weinstein are you going to also talk about remote sensing AI for ecology?
*Thread Reply:* thats where most of my research is, so just as an example of how we can use weak supervision on unlabeled data to tackle the ‘not enough data’ obstacle.
*Thread Reply:* As someone who is fighting to figure out how to keep doing this as a career, I can really second the lack of clear resources. Another issue, from the academic CS side, is the balancing act between impactful for conservation and publishable in CS that any CS grad student in this space has to consider when deciding what projects to focus on.
You guys, I made a random viz for anyone who wants to see: https://public.tableau.com/profile/tee8225#!/vizhome/TexasBirdsProject/TexasBirdBiodiversity?publish=yes I was using the eBird data to do a few visualization projects. One fun thing I did learn is that blue herons have a far wider range than I thought. I just moved to Texas so may use this data to see if I can catch sight of a few of them, haha! 🙂
*Thread Reply:* very cool. Did you download the csv or is live updating through the app?
Hi everyone - just wanted to introduce myself as a new member. I'm a Professor at the University of British Columbia, and I've been working on digital tech and environmental governance innovation for the past several years (my original training was in physics and environmental sciences). I'm currently working on a book on this topic to be published with Princeton University Press next year, and I also maintain a database of innovative digital conservation technologies (3,000+ and growing fast). Some of you might be interested in this meta-review of the academic literature on digital tech/environmental governance: Bakker, Karen, and Max Ritts. "Smart Earth: A meta-review and implications for environmental governance." Global environmental change 52 (2018): 201-211. https://www.sciencedirect.com/science/article/pii/S0959378017313730
*Thread Reply:* Awesome compilation!! Can't believe I didn't catch this paper when it first came out, thanks for sharing here!
I also collaborate with the United Nations Environment Program on their global initiatives on digital tech and environmental data. Here's a recent article I co-authored with David Jensen (UNEP) that outlines some of the ideas that are being debated by international organizations: https://medium.com/@davidedjensen_99356/digital-planet-20-priorities-3778bf1dbc27
*Thread Reply:* You should also post to Wildlabs if you haven't already!
*Thread Reply:* Thanks @Carly Batist I'll check it out!
*Thread Reply:* Weighing in from the climate change workshop, we'd love to have submissions from this group! If you do submit, remember to make explicit how your submission is relevant to climate change. Not every conservation-related problem is a climate change problem, but there can be overlap - for example if monitoring populations that are specifically under threat from climate change.
More info about the workshop here: https://www.climatechange.ai/events/neurips2020
*Thread Reply:* There’s two different kinds of recommendations I can give here. The problem of keeping up with the latest publications can be solved easily in computer vision by checking the following resources: • arXiv: https://arxiv.org/list/cs.CV/recent • CVF (all CV conferences): http://openaccess.thecvf.com/ As for picking the right pipeline: in all honesty, I deliberately don’t use the latest method. Oftentimes, they are not widely spread, very exhaustive to implement, and just an incremental improvement over existing methods. Instead, I tend to stick to the models that have been shown to be good. They may not yield the very best performance, but they are solid and can be adapted to one’s needs. For me, these include: • Image classification: I always use ResNet, even though there’s plenty of newer, more powerful methods. • Segmentation: I am a big fan of U-Net; it is easy to understand and performs like a champ. DeepLab is also great, though. • Object detection: I have been using RetinaNet and YOLO, if needed. Most of the time I could do away with bounding boxes and just predict points, in which case I use a ResNet without the average pooling layer. Honestly, this is what I would recommend for you. I may be biased, but I have been working with tiny objects from drones, and this outperforms other models in speed, accuracy and simplicity.
*Thread Reply:* I agree with this, I also think that its worth directing your focus away from architectures and more towards the entire pipeline. In ecology we almost always have limited data, getting familiar with fine-tuning strategies, transfer learning strategies generating synthetic data and understanding model parameters such as loss weights and learning rates often makes a HUGE difference as compared to the actual architecture.
*Thread Reply:* @Benjamin Kellenberger Thank you so much for your guidance! This is really helpful. For this project I have been using mostly DeepLabV3 and I was next going to try U-Net. I had not thought about using just points for these little carcasses instead of segmentation, but I'll give this a shot. Also thanks for providing these links! I'll keep an eye on them to stay current on the latest methods. I really appreciate you sharing your wisdom here.
*Thread Reply:* Thank you @Ben Weinstein! (I was actually just reading your recent paper in Ecological Informatics this morning.) I have been doing a lot of fine tuning with adjusting hyperparameters and transfer learning. I currently cannot use an HPC, so adjusting parameters and then re-training a model takes a really long time on the small server we have. Do you have readings you recommend for strategies to adjust the pipeline? I've worked through some examples in the past, but I'm certainly not an expert and a lot of my adjustments are arbitrary decisions that I make in the process.
*Thread Reply:* usually testing a small portion of your model is good practice, it has the added benefit of revealing any overfitting when you go from your small dataset to the full dataset.
*Thread Reply:* Keras has a few optomizers, but I also use the ones from comet.ml
*Thread Reply:* that helps with parameter searches.
*Thread Reply:* more broadly the focus on data generation is so important. I’ve seen transfer learning among very disparate classes work well, we used that tree model to start and trained for drone work on birds and it was hugely beneficial.
*Thread Reply:* there is only so much information you can squeeze out of small ecological datasets.
*Thread Reply:* I think the camera trap competitions that @Sara Beery ran also showed that grabbing imagery from other locations can be useful, even in the extreme situation when you don’t have real information for a target class (see their kaggle competition going from CA to Idaho)
*Thread Reply:* For parameter optimisation, I second @Ben Weinstein — trying on a decently sized subset usually reveals enough as to what settings one needs.
*Thread Reply:* Agreed with all of the above. I think it's good to start with something robust that has been proven out over time, and make sure you've really maxed that out by exploring data augmentation, hyperparameters, and cotraining or pretraining from other relevant public datasets. Once you have an idea of what you can do with a well-proven model it makes it easier to determine whether a newfangled method is worth trying to incorporate. I completely agree with @Benjamin Kellenberger that frequently the newest methods can be unreasonably complicated to implement and finicky to train for very small gain.
*Thread Reply:* besides our methods, right @Sara Beery
*Thread Reply:* those are always worth adopting.
*Thread Reply:* a follow up here from today. Forgot to apply per image normalization and centering. Added per image and performance increased 20%. Alot more important than any architecture difference.
*Thread Reply:* @Benjamin Kellenberger @Ben Weinstein @Sara Beery Thank you all so much for your guidance! I fiddled with hyperparameters, augmentation, optimizers, and loss functions and now I have good model (it's actually better than our technicians at finding objects. Now I'm getting started on another project and have another question, so I'll start a new thread.
Two new individual ID datasets available on lila.science thanks to the good folks at Wild Me:
http://lila.science/datasets/great-zebra-giraffe-id http://lila.science/datasets/whale-shark-id
Everyone go identify some animals with AI! (Bonus: there are lots of fantastic baby zebra pictures.)
*Thread Reply:* Looks very cool! Would it be possible to have some example images on the landing page for each? Not necessarily hard examples, but just to see the variety. I like how @Sara Beery did it for her Caltech Camera Trap dataset: https://beerys.github.io/CaltechCameraTraps/
*Thread Reply:* I do like that idea, in fact I like the idea of doing that across all LILA datasets, but then again I'm also lazy, so let's call it a crowdsourcing effort. 🙂 If anyone sends me a bunch of images from any dataset on LILA that demonstrate the breadth of the data in an interesting way, I'll find a way to either put them right on the main page for that dataset, or - more likely - host a separate HTML file somewhere that shows those images, and link to that from the main page.
Hi there! I've been lurking for a while but haven't introduced myself yet. I'm a PhD student in U Washington's ICTD Lab, mostly focused on technologies for the community/social/political aspects of conservation.
It's not really AI, but @Kennedy Murrithi, @William Njoroge and I have a new paper about basic mobile phone services for engaging with Ol Pejeta's surrounding villages: opening communication channels to improve service delivery and more fairly share the costs/benefits of conservation, and building social capacity to work together better when issues arise. (To toot my own horn, we won best paper at ACM Computing and Sustainable Societies this year!) Check it out:
The paper: https://mattziegler.net/papers/Ol-Pejeta-Phones-COMPASS2020.pdf 15-minute presentation: https://www.youtube.com/watch?v=hEUJq97n1-E 3-minute version: https://www.youtube.com/watch?v=gRryk_xUMkk
*Thread Reply:* Nice! I've definitely heard of you. Maybe catch you there again sometime!
*Thread Reply:* I recently annotated bounding boxes in video using vatic, which is open source, but I'm not sure if it supports zooming. http://www.cs.columbia.edu/~vondrick/vatic/
*Thread Reply:* I just hosted it locally, and didn't do a large amount of data.
*Thread Reply:* i heavily use rectlabel for Mac
*Thread Reply:* Anyone have experience with VoTT? Is this what is used for the Azure labeling tool?
*Thread Reply:* There’s an open source project called labelme, and I’ve used it for a project earlier. Does video annotation well, but I will have to check if it has a feature for zooming in.
https://github.com/wkentaro/labelme
*Thread Reply:* I've started using CVAT in the past few months and have really liked it. Developed by intel but open source has nice interface and supports most standard types of annotation on images and videos. Used to be a little unstable but much better now. Also easy to run on a server so many people can annotate same project. Allows zooming. https://github.com/openvinotoolkit/cvat
*Thread Reply:* After looking into this question, I ended up using VoTT for ease of use and the ability to export records to different formats.
I’m giving a talk on “A Computer Vision for Ecology” at Florida Museum National History (+ iDigBio) at 3pm ET today. Zoom webinar: https://ufl.zoom.us/j/98489487554?pwd=dW1ULzZyTnFoQVYrczJja1JKMDV3dz09 Meeting ID: 984 8948 7554 Password: 671049. I was asked to do a wide ranging talk, so @Siyu Yang @Sara Beery @gvanhorn, @Holger Klinck and others get a shoutout for recent work. Thanks everyone who helped with the slides and address those big questions. Slides are here if interested. https://www.dropbox.com/t/GqyTNrallTfy7G9v
*Thread Reply:* If anyone actually wanted to see my talk on computer vision for ecology. Here is the recording https://ufl.zoom.us/rec/share/OCn_21P2mqZa2jAqy3xNJ8xXRbqgXuArqekIssggyTsn2Zvn5ccv9z1QrifnamkM.eqhoPnR_XS57QOMJ
*Thread Reply:* Definitely. Do you happen have a slide that shows/plays any audio and birdnet predicts it? I just show a screenshot and describe it.
*Thread Reply:* Feel free to modify as you see fit!
This is cool, lots of tropical forest data about to become public: https://www.planet.com/pulse/planet-ksat-and-airbus-awarded-first-ever-global-contract-to-combat-deforestation/?fbclid=IwAR3BwzSlsuKrFZbdDwJ3t68nRxX6fibSjFVaW7QWUsc39wNGx9_kLwVwJDc
*Thread Reply:* I had a meeting with Global Forest Watch not so long ago and didn’t hear about this @John Brandt will you at WRI have access to these data? Global Forest Watch is explicitely mentioned in the announcement. Will there be a web portal or an agreement with WRI? What can we expect in terms of data access for the scientific community? Like if, for example, wanted to host a server performing deepforest predictions for ecuador in realtime.
*Thread Reply:* @Ben Weinstein that sounds amazing
*Thread Reply:* We have a call with Planet and NICFI this afternoon to discuss the terms of the data release. We know the contract is for 2 years, with an option of extension up to 4 years, and we know that data will be released one month after the mosaic period (so November data will be released in December). We don't yet know what the process will be like for getting access to the data - we believe it will be locked behind some sort of application process, but it will be free for non profits / academics to use if they go thru the process. I don't know when they are releasing this to everyone, though I know it is planned. For the moment they're working with us at WRI and people at FAO to user test the data access
*Thread Reply:* thanks. Keep up informed. Let me know if I can be of use.
We are putting together a proposal for the Fine-Grained Visual Categorization (FGVC) workshop at CVPR 2021, with a deadline of Oct. 16. (See https://ai.googleblog.com/2020/05/announcing-7th-fine-grained-visual.html for a recap of the most recent FGVC.) We’re looking for a o-organizer for the 3rd edition of the Herbarium challenge, with data provided by New York Botanical Garden. Kiat Chuan Tan served in this role the past two years, and kindly offered to give pointers to/share his codebase with the next co-organizer.
If this might be of interest to you, drop me a line and I’d be happy to tell you more about what it entails!
*Thread Reply:* Do the separate RGB and hyperspectral models have a lot of overlap in their errors on test sets? Are the hyperspectral errors basically a subset of the RGB errors?
*Thread Reply:* not yet. We have a third set of validation data. First trying on the same holdout set used to train each member of the ensemble. It should ATLEAST overfit and improve, even if we later find out it doesn’t generalize to a new test set.
*Thread Reply:* I’ve tried for example, batchnorm before concat because maybe the networks have learned features of different magnitudes.
*Thread Reply:* i’ve tried multiplying them instead of concat.
*Thread Reply:* https://github.com/weecology/DeepTreeAttention/blob/9b48d277df8024286a41e86100d6ad5e180ca156/DeepTreeAttention/models/Hang2020_geographic.py#L105 to make it literal.
*Thread Reply:* i’ve tried freezing the layers versus finetuning with the new joint loss.
*Thread Reply:* Yeah you're right to just be looking at the holdout sets first that you used to train them both. My question is: with that holdout set, are the sets of examples that the individual models get wrong subsets of each other?
*Thread Reply:* are there a large number of specific examples that RGB gets right than hyperspectral gets wrong?
*Thread Reply:* don’t know yet, i’ll look, i’m sure they aren’t perfectly nested, but I hear you. RGB data (3 channels) + infrared (369 channels), so atleast they should be reasonably diverse.
*Thread Reply:* its funny because I assumed I’d get a modest improvement and the real goal was to study joint connections at differing layers so that the two data sources could teach each other features. I haven’t anticipated this going so badly.
*Thread Reply:* I've had this problem before in other contexts and it just turned out that they were not orthogonal enough
*Thread Reply:* Another thing you can do is basically freeze both pretrained models, and initialize the final weights on the hyperspectral half of the concat to exactly what it is on the pretrained model, and the RGB half to 0. So you're literally starting with exactly the same individual hyperspectral model
*Thread Reply:* Then train the last layer and see if it improves at all in the first few batches
*Thread Reply:* Another thing might be to see the relationship between the number of channels you include in the hyperspectral model and the performance. With hyperspectral imaging its always possible there is so much overlap that having more than a couple dozen channels basically doesn't add anything new, and that could extend to RGB also.
*Thread Reply:* I was doing a wetlands mapping project a few years ago where I took a different approach to this question. I had access to 8 band orthophotos, but also to several GIS layers, which had information about soils, vegetation type, and land cover classification. I ended up using more standard machine learning models (Random Forests, Boosted Regression, MARS) and created a training set of points across the landscape, which I classified as "wet" or "not wet" in roughly equal proportions. Then I used these points to "drill down" through all the different layers and ran those as predictors for my model. The hyperspectral layers were still the most informative predictors for water, and if you were going to use something like "soil type" as a predictor, then you need to make sure you fit the model using all classes you want to predict back to (otherwise the model chokes on new classes it hasn't seen yet when it goes to predict - why the numerical inputs are better than categorial) anyway, just figured I'd share, because the models actually did a really good job of predicting wetlands on the landscape ..even finding those covered with vegetation and these very small anchiline pools (few meters wide only). It was pretty compute intensive, especially when it came to the prediction step (for all Hawaiian islands on 5m grid) but certainly manageable with modern and cloud resources. The project is here, on github if you want to check it out. My code was pretty clunky back then, but you'll get the idea.
*Thread Reply:* Welcome! What lab at CONABIO works on these issues? I work with researchers throughout latin america, mostly ecuador and colombia, and we don’t have any partners in mexico yet.
*Thread Reply:* hi, thanks!. I work at the Dirección General de Proyectos Interinstitucionales, where do you work?
*Thread Reply:* I’m at University of Florida, but work closely with non-profits on biodiversity surveys. https://deepforest.readthedocs.io/, http://benweinstein.weebly.com/deepmeerkat.html. I’d be interested in hearing more about what the status of these kind of work is in mexico. The audio work is for bird detection?
*Thread Reply:* Hi, Ixchel. Welcome to this wonderful channel. I will also be very interested in learning more on work on these topics in Mexico, as my research group also works on similar topics at the USC Center for AI in Society.
*Thread Reply:* We have a project called Sipecam that involves monitoring techniques to evaluate the effect of defaunation. If you want, we could have a meeting to talk about it a little bit more with other part of the team. https://sipecamdata.conabio.gob.mx/
*Thread Reply:* @Elijah Cole (Deactivated) do you have any tips here?
*Thread Reply:* I've never worked with hyperspectral imagery so this is total guess work...
Have you tried determining feature importance at the band level (then filtering out some bands) -- 369 dimensional inputs to a CNN seems too large. Are the spectral ranges of the bands adjacent to one another, maybe you could merge groups? Is spectral attention over input bands a thing @Elijah Cole (Deactivated)?
You could apply dropout over channels between input and first conv layer (Similar motivation to cutout augmentation, but channel wise)
*Thread Reply:* thanks, following a recent spectral/spatial HSI paper on architecture. https://arxiv.org/abs/2005.11977 still trying to find the right place for class imbalance. I will be explore band reduction too. Trying to debate whether its worth going down of the road of synthesizing images using a GAN. https://arxiv.org/pdf/1903.05580.pdf
*Thread Reply:* Relevant (and very recent) paper that I just stumbled on https://arxiv.org/pdf/2010.12337.pdf
*Thread Reply:* I find that the effective number of samples works better than normal loss weighting methods: https://arxiv.org/pdf/1901.05555.pdf
*Thread Reply:* Would it make sense to do gaussian blur augmentations, i.e. run a gaussian filter over the spatial domain? Thinking about this, there might be some sophisticated augmentation that simulates atmospheric noise? This wouldn't address class imbalance, but rather "uniform" data augmentation.
*Thread Reply:* A friend just recommended ATRAN (https://atran.arc.nasa.gov/cgi-bin/atran/atran.cgi) to sim. atmospheric noise. Might be most relevant for satellite rather than aircraft data, depending on the height that you took your data.
*Thread Reply:* A warm welcome to the Kiwiland! Having a different background than CV is more helpful than it first seems—it’s the combination and transgression of disciplines that makes conservation fly in my opinion.
*Thread Reply:* honestly what we need is more backend engineers. We’ve got great models. how to share them, make them reproducible, create competent UI interfaces for less experienced programmers. Good data storage and metadata management.
*Thread Reply:* Those are the things we need.
*Thread Reply:* on my todo list is to have a look at https://www.biorxiv.org/content/10.1101/2020.10.02.323329v1 and see about all the above things
*Thread Reply:* @Ben Weinstein That's really encouraging to hear. I have plenty of experience with that kind of stuff and will be happy to help. I will read through the paper you sent over as it's actually something I have been thinking about already.
*Thread Reply:* Welcome @Petar Gyurov. This is a great place to learn from the experts. I'm an ecologist by training and I'm very fortunate to get great advice from these brilliant computer scientists.
*Thread Reply:* Thanks @Mikey Tabak I've been taking a look at some of your work (MLWIC) -- great stuff 👍
*Thread Reply:* No, sorry. For segmentation we used a commercial product http://www.imageseg.com/
*Thread Reply:* @Caleb Robinson has some good work on land cover mapping... his CVPR 2019 paper has released code. It's in tensorflow, not pytorch, but may still be useful!
code: https://github.com/calebrob6/land-cover
*Thread Reply:* oh true! That repo is really amazing. Unfortunately, our whole code base is in pytorch and I'd need to access the weights of the model at different layers. So it seems quite tricky to integrate. But i'll definitely give it another look, thanks Lily!
Thanks for the link Ixchel! This might actually be interesting for another project/idea. Seems like I can upload an image to test the quality.
*Thread Reply:* Hmm all my stuff is in Keras and nothing pretrained on RGB imagery. Though @Ben Weinstein may know of something good!
*Thread Reply:* Thanks for the vote of confidence @Björn Lütjens! I'm actively working on a similar repo in PyTorch with which I'll train similar LC models -- I'll ping you in a few weeks when this is done.
*Thread Reply:* that sounds amazing! thank you so much @Caleb Robinson
*Thread Reply:* @Björn Lütjens I just noticed this thread in my thread list -- forgot all about it, sorry! The repo in PyTorch is here https://github.com/calebrob6/dfc2021-msd-baseline. We use it for training baseline models for the data fusion contest, however it works "as is" for training with high-res labels too
*Thread Reply:* amazing; thank you so much Caleb!! :D:D
*Thread Reply:* Perhaps of interest when using "top" and "side" views: https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Wegner_Cataloging_Public_Objects_CVPR_2016_paper.pdf
*Thread Reply:* Thank you @Oisin Mac Aodha, I'm checking this out now
*Thread Reply:* How big are the birds in each image stream? If the size is very different it might be useful to have a pipeline of (1) two different bird detection algorithms for each image stream, (2) crop and resize, s.t., the birds in each small bird tile are same size, (3) use bird classification algorithm trained on both image streams; maybe even combined with ImageNet etc.
I'm totally not an expert here, so just brainstorming 🙂
*Thread Reply:* Thanks @Björn Lütjens. I'm not sure how large (how many pixels) the birds will be (I should be receiving some images soon). But another challenge is that of the overhead images, they will be taken from different elevations, so the birds will be of different size. I like the idea of having a preliminary detection model to pull out the bird, standardize the size, and then run a classification algorithm. My major concern with combining the two types of datasets, though, is that birds are going to look different from overhead vs from the side, so I might need a different classification model for each type of view also.
*Thread Reply:* agreed! do you have sample images? 🙂 🐦
*Thread Reply:* Unfortunately no sample images yet. I'm also not sure I'll get to share them when I do
Vaquita Hacks hackathon is looking for mentors!
"The Vaquita is a critically endangered species, and time is running out to save it. Join Earth Hacks, The Conservation Project International, The Countering Wildlife Trafficking Institute, Earth League International, and many more for Vaquita Hacks on Dec 12-13, the first hackathon dedicated to vaquita conservation! Projects from the hackathon will be going directly to organizations working on vaquita conservation."
• They are looking for mentors to help student participants in this program. Folks who are interested in mentoring should email Sanjana Paul <sanjana@earthhacks.io>. • Applications for student participation are open (now extended until November 23).
*Thread Reply:* @Sara Beery has some work that might be useful: https://arxiv.org/abs/1912.03538
*Thread Reply:* Happy to chat and look at some data and see if something like our attention- based approach or a more traditional detection/tracking method might make sense!
*Thread Reply:* Thank you @Sara Beery and @Elijah Cole (Deactivated)! I hadn't thought about using Sara's Context R-CNN, but this might be the best strategy. I'm going to keep trying to develop a pipeline combining deeplab and a 3d-cnn as I described above, but I don't think this will work. I appreciate your willingness to chat Sara. I'll reach out next week.
*Thread Reply:* Hi all - I'm new here, but am interested in this conversation for a few reasons. We have a thermal videography study funded for small bats (Pacific Sheath Tails) on Aguiguan in the Marianas. I think my colleagues at USGS in Colorado have been using a differencing algorithm to detect bats in video, and can catch up with them to see where they are at with that, per your question. But I also just got a project monitoring Boobies in Hawaii with a long series of mostly empty camera trap data, where there are intermittent Boobies, but fixed decoys in the images, so I think anything I train to pick out a booby is going to also pick out the decoys, so I need a way to pull them out and only select photos with the real boobies in them. I am interested to read Sara et al.'s paper and have a flexible schedule if there's any way you'd be willing to let me sit in on your conversation?
*Thread Reply:* We already talked, but re: the boobies...if the cameras are static can you just label where in pixel space the decoys are and throw out any detections with high IoU with the decoy locations?
*Thread Reply:* And I'm sure @Mikey Tabak would love to chat with your colleagues at USGS, that sounds like a very similar challenge!
New aerial image dataset (~21k point annotations on seabirds) added to LILA:
http://lila.science/datasets/aerial-seabirds-west-africa/
Hi there, my name is Jenna. I am a User Experience Designer at Vulcan Inc. supporting a project on dolphin signature whistle annotation. I am interested in gathering more information and gaining a better understanding about the marine mammal bioacoustics community, specifically acoustic dataset processes and annotations.
All information collected will be used for research purposes to help inform how we can best support this community. The survey should take around 10 minutes.
Thank you for taking the time to fill out this survey. Your input is greatly appreciated! Please feel free to share this survey.
https://forms.gle/jvRXh3kjYHKPqUZ78
Cheers, Jenna, User Experience Designer at Vulcan Inc.
*Thread Reply:* @gvanhorn this reminds me a bit of some of what you were talking about this morning about hard/unlabeled stuff in your audio data, any thoughts?
To me I guess an important first thing to think about is what types of errors would be worse in this scenario (overpredicting birds vs. missing birds), and if you can have human experts address the challenging cases?
*Thread Reply:* I think it's worse to miss birds, since the ultimate objective is to develop models to identify them to species level if possible and count them, and my other fear is that they look so similar, it will actually confuse the model (resulting in worse performance) as it tries to distinguish between the two.
*Thread Reply:* I also was a little concerned when I read that YOLO models sometimes don't perform as well on small targets (which is obviously what we have here), so if anyone has used these models specifically to do a similar project, I am interested in your results.
*Thread Reply:* This is a drone image, do you have a camera trap example? Are you thinking of training a joint model or separate models for the camera traps and the drones?
*Thread Reply:* In my experience with trees, its always better to include everything, or else you risk training your detector to ignore potential objects that look similar to the objects you are trying to predict. Depends a bit whether recall or precision is more important for your question.
*Thread Reply:* I am also working on data preprocessing using deep autoencoders where you first screen your data for potential annotation anomalies.
*Thread Reply:* so my recommendation is include, since you can always work to throw out later. You can never recover annotations you don't mark.
*Thread Reply:* In the dlib labelling tool, ImgLab, you can mark regions to ignore. The dlib object detector training (MMOD) will not use these boxes for training, and it will also not use these areas for negative examples. I'm not sure if Yolo ot other object detectors have a similar mechanism.
*Thread Reply:* @Sara Beery the camera traps and drone images are different projects. I am not aware that we have any co-collected data of these different types.
*Thread Reply:* @Ed Miller and @Ben Weinstein thanks for the feedback, interesting.
*Thread Reply:* It may depend on what your purpose is. I’m working on object recognition for aerial survey images, the purpose being to derive population estimates from counts of animals on transects. Since we need to assess the bias, precision and accuracy of our population estimates, there is no substitute in our case for hand-checking a large set of randomly-selected images so that we can estimate both a false-positive and a false-negative (missed animal) rate for the neural network model. For jobs where it isn’t as important to understand bias, precision and accuracy, you may be able to get away without manual checking. Multi-model comparisons, Bayesian updating, or other clever automated methods may be able to replace hand-checking in some cases, but human observers are still the gold standard when you’re on the wild frontier and the abilities of models haven’t been thoroughly assessed.
*Thread Reply:* @John Payne what are you working on? We should chat, sounds basically like the project we have in the everglades on wading bird population dynamics from drone surveys. http://tree.westus.cloudapp.azure.com/everglades/
*Thread Reply:* Hi @Ben Weinstein, I have two survey projects; one with Howard Frederick on aerial surveys in Tanzania and another with a colleague who is using drones in Kazakhstan. I look forward to talking — it would be great to get a conversation going around these issues.
*Thread Reply:* @Ben Weinstein @John Payne I would love to chat with you both about methods and study systems. Although we are working towards using drone surveys of breeding birds on Lehua, our refuges program has a drone and there is just alot of interest in using this technology to conduct surveys. An intro meeting would be great.
*Thread Reply:* @Benjamin Kellenberger is my go-to expert on animal surveys in drone data, I'd recommend bringing him in to the meeting as well!
*Thread Reply:* @Mari Reeves I look forward to hearing about your project. @Sara Beery That’s a good idea. We’re in communication with Beni and have been experimenting with his AIDE program. I’ll send out an email to organize a call.
*Thread Reply:* Hey all - wow great timing. I work for The Nature Conservancy and we’re about to embark on an aerial data labeling project (sea bird census around Palmyra). We’re currently trying to evaluate tools for processing and labeling large geoTIFF orthomosaics; I hopped on this channel to see if anyone had any recs for workflows/software to assist and this conversation has already been super helpful. I didn’t know about AIDE or ImgLab. Does anyone have any other recommendations for drone data labeling tools we should check out?
*Thread Reply:* @Nathaniel Rindlaub are you looking for automated processing or manual or a mix thereof?
*Thread Reply:* @Howard L Frederick we're looking to develop a model that will assist in a manual review process, so a mix I suppose. However we're just in the early stages of this and are trying to figure out a workflow for building up a training dataset at the moment.
*Thread Reply:* We have a private Zooniverse project up and running for everglades wading birds.
*Thread Reply:* Works great and is easy.
*Thread Reply:* also, btw, starting from a pretrain model is super useful for transfer learning. For the birds we start from a tree model. Shameless plug for https://deepforest.readthedocs.io/en/latest/
*Thread Reply:* and once the first paper comes out i can share the egret label model, which should be useful for other nesting birds.
*Thread Reply:* @Ben Weinstein cool! Very helpful - I'll be sure to check out Zooniverse, and I'd love to try out the egret label model when it's ready. Also interesting that a model trained to detect trees is a good place to start for finding birds sitting in them!
I am just getting up to speed on our project (@Vienna Saccomanno can speak to all of this in more detail) but I believe we are trying to work towards a (1) a detection model that can pick out birds both roosting in palm trees and floating on the water... and then (2) likely a classifier that can classify them as either blue or red footed boobies.
*Thread Reply:* @Ben Weinstein this is incredibly helpful and interesting. For our work on Palmyra to census four different species of seabirds, we are mostly focused on the birds in the canopy (mostly Pisonia Heliotropium) and I am intrigued at your use of DeepForest. We've been concerned about animals partially covered by the canopy, so I'm very much looking forward to your first paper. Automated processing is the goal, but we are aware that some level of manual review might be necessary.
*Thread Reply:* I'm a pretty strong believer that all AI for conservation projects should have some built in, ongoing manual review, if nothing else than to verify difficult data and do quality control! My hope is that process would be much more lightweight than reviewing all the data, but it would help catch any obvious issues and make sure your model is maintaining accuracy over time
*Thread Reply:* Thanks for that, @Sara Beery. I think that is right and looking forward to learning more.
*Thread Reply:* @Nathaniel Rindlaub I processed a few big orthos not long ago using ImageMagick to cut the image into tiles then giving annotators folders of images in labelImg; I’d probably do it in CVAT if I repeat the process since you can manage multiple users, and overlap images between individual annotators to check consistency.
*Thread Reply:* @Howard L Frederick amazing! Thanks for your recs! I'll definitely check them out.
*Thread Reply:* @gvanhorn has some awesome work on how to intelligently & efficiently combine annotations (including bboxes) from sets of annotators. I used it when building the Caltech Camera Traps dataset. https://openaccess.thecvf.com/content_cvpr_2018/html/Van_Horn_Lean_Multiclass_Crowdsourcing_CVPR_2018_paper.html
*Thread Reply:* @Nathaniel Rindlaub there are some reviews of annotation software linked here: • https://medium.com/tektorch-ai/best-image-labeling-tools-for-computer-vision-393e256be0a0 • https://lionbridge.ai/articles/image-annotation-tools-for-computer-vision/ • https://hackernoon.com/the-best-image-annotation-platforms-for-computer-vision-an-honest-review-of-each-dac7f565fea
*Thread Reply:* … however there is a wide variety of features and I have yet to find one that does everything I want. Some don’t zoom, most don’t deal with teams, some have very heavy web/cloud requirements (vs. in-house) …
Hi all, thrilled to be here! Thank you @Dan Morris for the kind invite. We work with snow leopards, and recently partnered with Microsoft who developed a cool tool for us to scan through thousands of images of snow leopards from camera traps for analysis using spatial capture recapture. Here to learn and explore the rapidly growing overlap between mathematical statistics, computer science, ecology and biology, and technology! My 30 seconds' worth of claim to fame has been the opportunity to feature on MS AI ad last year 😊: https://www.youtube.com/watch?v=68WheTADA-g&t=7s
*Thread Reply:* Nice to meet you, and great video! We're working on individual ID for snow leopards with Pose Invariant Embeddings. Would love to talk further.
*Thread Reply:* Hi @Jason Holmberg (Wild Me), heard about your team from Dan. Let me know please when a good time to connect!
*Thread Reply:* Yeah, autoencoders are a cool way to go for finding outliers in the dataset as a whole. For identifying class label mistakes, there are lots of adhoc tactics out there (which you have probably thought of) all of which require some brute force examination of the data. Cross validation and looking at the entropy of the predictions coming from a classification network is one standard tactic. Essentially like an active learning setup.
*Thread Reply:* Okay, so it would be a bad idea to concat the actual class to the autoencoder and try to get it to predict both the image and the label?
*Thread Reply:* Maybe worth a shot. Depends on how much time you want to spend on the problem/coding/debugging. If you have prior knowledge about which classes might get confused, it might be quickest to just browse through that subset looking for mistakes. With enough data, I typically embrace the noise in the training set and worry more about the quality of the evaluation set.
*Thread Reply:* thanks. I appreciate it. Its an interesting problem because its not traditional mislabeling in the sense that a annotator thought it was a sharp-shinned hawk and it was a cooper's, but more subtle. Ground truth is captured by people marking the tree stems with points. So there is alot of guesswork about what tree crowns are visible for a canopy. In this example, we have our detection algorithm (blue box) predict one tree, but there are two species ground truth underneath. One correctly predicted, one incorrectly predicted. Obviously because there is only one crown. Looking for an algorithmic way to anticipate these sceneries. But yes, hand review!
*Thread Reply:* Is there a box associated with each dot? (one-to-one?)
*Thread Reply:* nope! ground truth gets a point. The box is predicted from my tree detection model. Fun, right?
*Thread Reply:* so you have to choose with pixels go with each point for training.
*Thread Reply:* hmm, interesting. Your ultimate goal is boxes or points?
*Thread Reply:* that's an interesting question. boxes i suppose, since that entire thing is a tree.
*Thread Reply:* no one has ever asked me that before.
*Thread Reply:* No idea if this is helpful, but have you seen this work? https://arxiv.org/abs/1602.08405 https://arxiv.org/abs/1708.02750
*Thread Reply:* Not sure if it is related, but your example figure reminded me of those works
*Thread Reply:* totally, thanks for the ideas.
*Thread Reply:* @Sara Beery I see you lurking on my tree problems.
*Thread Reply:* But also I'm running into the same issues so I'm hoping you just solve them for me ☺️
*Thread Reply:* Ben I think your question of whether “this black oak is different from all other black oaks” is an important one. This is kind of an inelegant, brute-force suggestion, but how about running the autoencoder on one class at a time (I mean in addition to running it on the whole dataset)? Would that be prohibitively expensive?
*Thread Reply:* I was actually kinda thinking the same @John Payne but it might be the case that you wouldn't have enough data per-class?
*Thread Reply:* i mean, definitely not enough data. But no, easy to do, no computational challenge, we have enormous HPC resources. I'm not entirely sure its different than reconstructing all images, plus the label. So your autoencoder has two heads and you take the joint loss.
*Thread Reply:* I guess that is probably theoretically similar, and might help with the lack of data, but in the per-class sense you're letting the autoencoder learn only relevant features for reconstructing a specific class, which might let it separate out the clusters a bit more? As opposed to needing to learn to reconstruct all species?
*Thread Reply:* i'll give it a try today. writing it now.
*Thread Reply:* I had another idea; not sure if it’s any good. You may have seen this recent paper on anomaly detection: https://arxiv.org/abs/1911.10676. They train an autoencoder to restore degraded images (rotated or color loss), which forces it to learn semantic features of the class (they run it on one class at a time). Then during inference, normal (non-anomolous) images can be restored properly but anomalous images that are missing the proper semantic features will not restore properly and so they cause large restoration losses. Their method performs extremely well on several anomaly detection tests.
So: I wonder if you could use that method, but also include the label as part of the input: basically pass a one-hot encoded vector of class labels to the autoencoder along with the image. During training, it would learn to tell the classes apart and how to restore them. Then during inference, if the image was a good example of the class, presumably it would restore properly, but if it was labeled wrong then the model might choose the wrong set of features to restore. I suppose you might need a fairly deep model if you had a lot of classes to learn, but in the paper they mention a Resnet 34, so perhaps it’s not out of reach?
*Thread Reply:* Welcome indeed! 😄
*Thread Reply:* Great to have another person with a science background getting into the AI/ML stuff! We can commiserate together 😅 though I work with mammals not birds..
*Thread Reply:* Our pipelines uses Agisoft for orthorectification, Zooniverse for annotation, deepforest for transfer learning (https://deepforest.readthedocs.io/), R shiny for web dashboard (http://tree.westus.cloudapp.azure.com/everglades) hosted on azure VM instance.
*Thread Reply:* Very interesting discussion - needed another 4h or so …
*Thread Reply:* Hi Ben, Thanks a lot for making the meeting of yesterday happen! It was great to see and get to know you all. Here’s some links to my works:
- Detecting animals in drone images and using as much of the background for training as possible: https://arxiv.org/pdf/1806.11368.pdf
- Weakly-supervised object detection (“image has/doesn’t have animals, localize them for me”): http://openaccess.thecvf.com/contentCVPRW2019/papers/EarthVision/KellenbergerWhena[…]ifferenceImprovingWeakly-SupervisedCVPRW2019_paper.pdf
- AIDE paper: https://besjournals.onlinelibrary.wiley.com/doi/pdfdirect/10.1111/2041-210X.13489
- AIDE GitHub link: https://github.com/microsoft/aerial_wildlife_detection
*Thread Reply:* Hey Katie, I assume you are working Laura Prugh at UW? If not that project is really similar!
*Thread Reply:* Hey Ben, Yes that's the one! I am Laura Prugh's student. Currently the model is a (pretty simple) custom network, but I am looking for ideas to improve the accuracy. Would love to connect after the holidays if you have time!
*Thread Reply:* Hi @Nathan Hahn, I responded to the survey. As a technologist with no formal conservation background, I found many of the questions were not applicable for me. Many were more suited to the conservation scientist I collaborate with.
*Thread Reply:* Hi @Nathan Hahn great topic - the ecologist / engineer interface is a really important often often blocking link. Keen to hear more.
*Thread Reply:* Dimitris Samaras and I collaborate on a number of problems in the area of computer vision and conservation, might be worth considering doing a PhD with him (https://www3.cs.stonybrook.edu/~samaras/). His lab does a lot of interesting things applying computer vision to both health and environmental applications.
*Thread Reply:* Thanks for the suggestion @Heather Lynch, I'll check his lab out!
*Thread Reply:* @Armin Bazarjani we are working with on a suite of projects and would love to chat if you are looking for projects to gain some experience in sustainability and conservation.
*Thread Reply:* That’s awesome, excited to see what you do!
*Thread Reply:* @Sarra Alqahtani in January, let's talk about your needs and how we can be helpful. My lab works on tree detection/species classification. Mostly in temperate zones, but have a paper in review in French Guiana. I was just talking to a colleague in the Ecuadorian amazon on potential applications there. See recent pubs for links, python packages, https://scholar.google.com/citations?hl=en&user=7POnELAAAAAJ&view_op=list_works&sortby=pubdate, https://github.com/weecology/DeepTreeAttention, http://visualize.idtrees.org/
*Thread Reply:* also @John Brandt and @Tony Chang for their experience on multi-temporal satellite fusion for land cover + deforestation analysis. https://www.mdpi.com/2072-4292/11/7/768
*Thread Reply:* We also are just starting (I can't get planet to return my emails!) work on UAV satellite fusion for wading bird colonies. Would love your input here, as the project is just forming.
*Thread Reply:* Thanks for the intro @Ben Weinstein! Congrats Sarra on the NASA grant! Definitely interested in helping, I’ve been working lately on a US scale forest structure project, but could scale globally pending more data. We are focused on using Harmonized Sentinel Landsat data and fusing to other datasets. Haven’t looked into UAV data much, but would be interested to learn more.
*Thread Reply:* @Ben Weinstein @Tony Chang Thank you so much for the information! Would love to meet with you both to discuss the overlaps between our research.
*Thread Reply:* great, i'll organize something for January. Keeping @John Brandt and anyone else in mind as well. We did a recent open meetup on drones for wildlife monitoring and i think it was useful. Let's post agenda items of interest here. Immediate thoughts looking at that postdoc ad: 1) introductions, goals, and background, 2) integrating multi-sensor imagery for deep learning classification, 3) training multi-scale (resolution) classification workflows, 4) available datasets for benchmark and annotations. Feel free to add.
*Thread Reply:* I would include land change/deforestation detection using time series data as unsupervised way of classification.
*Thread Reply:* Okay everyone, let's try to set up a meeting in the next couple days. Anyone is welcome to come discuss the topics listed above, I think @John Brandt @Tony Chang @Sarra Alqahtani are interested. https://whenisgood.net/4k4kphn
*Thread Reply:* Thanks for setting this up @Ben Weinstein!
*Thread Reply:* Hey! Would love to join this. We are working on a massive tree planting project with Mastercard (https://www.mastercard.us/en-us/vision/corp-responsibility/priceless-planet.html) and are starting to do UAV classification work as well as change detection with medium res imagery
*Thread Reply:* bump here to fill out times @John Brandt and others
*Thread Reply:* Sorry for the late action. Just sent my available times
*Thread Reply:* just confirming the time zone worked for you, tony and I are both west coast, it should have auto show you the correct timezone, I set it to PT. Did you mean 6pm ET?
*Thread Reply:* okay, just @John Brandt
*Thread Reply:* I added times in as well, I won't have much to add but I'd love to learn from you all :)
*Thread Reply:* okay, @Sara Beery, @Tony Chang, @Sarra Alqahtani for the 4 of us Friday 1:30pm PT (tomorrow). I will email John, perhaps he isn't as on slack as we are. I think his feedback will be extremely useful, since he's basically devoted the last few years to this, plus i'm anxious to hear about the status of the WRI/Norway/planet tropical data program.
*Thread Reply:* Hey @Ben Weinstein, sorry I'm on vacation right now so I haven't been on Slack much -- I'll be back in the office next week though and I'll catch up with you
*Thread Reply:* no problem, when you get back drop me a note. I'll see if florence is around.
*Thread Reply:* okay, looks like she left WRI.
*Thread Reply:* Benjamin Weinstein is inviting you to a scheduled Zoom meeting.
Topic: Benjamin Weinstein's Personal Meeting Room
Join Zoom Meeting https://ufl.zoom.us/j/4610898573
Meeting ID: Dial by your location
Join by SIP
4610898573@zoomcrc.com Join by H.323
162.255.37.11 (US West)
162.255.36.11 (US East)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia)
149.137.40.110 (Singapore)
64.211.144.160 (Brazil)
69.174.57.160 (Canada)
207.226.132.110 (Japan)
Meeting ID: Join by Skype for Business
https://ufl.zoom.us/skype/4610898573
*Thread Reply:* Thanks for this meeting! It was great meeting you all!
*Thread Reply:* @Sarra Alqahtani I’m having a conversation with Caleb Robinson who was working on land-use change questions in the US and he works with a combination of MODIS and NAIP data. Most of what he does is applying epitomic representations for low resolution labels to work in conjunction with high resolution images. Your problem is kind of the reverse case (high resolution labels with low resolution images), but I’ll see if he has any novel solutions. I agree, GANs may not be the best way…but maybe?
*Thread Reply:* Thanks everyone. It was a very informative meetign. @Tony Chang We are still adjusting our GAN model using downsampled drone images and hopefully we get better results.
*Thread Reply:* @Sarra Alqahtani see the small blurb in there about gold mining https://maaproject.org/2021/norway-agreement/ this was the dataset I was mentioning, connected to @John Brandt’s work.
*Thread Reply:* @Sarra Alqahtani, we have a remote sensing machine learning zoom group for biology.
Hi all. For the past couple of months I have been working on a project involving MegaDetector that I am excited to share with you.
I have built a GUI around MegaDetector that speeds up the process of weeding out empty camera trap images. The application detects animals in your photos, then lets you review the output of the model and make corrections. At the end your original images are sorted in folders. It's designed to be relatively simple to use.
I have just published version v0.0.1-alpha which you can download from https://github.com/petargyurov/megadetector-gui/releases
If you don't have time to download it and try it, here is a short demo video: 📽️ https://streamable.com/h5pcdu
My next goal is to implement more models and the ability to train a custom model, as well as few other things. Let me know what you think!
*Thread Reply:* Also, for this project, I did some refactoring the base MegaDetector code, mostly just to improve usability. If you want to use MegaDetector as a CLI or as part of your scripts, you may want to look at my other repo https://github.com/petargyurov/megadetector-api
*Thread Reply:* Have you thought about adding the ability for the reviewer to box animals that weren't found initially? Might be useful for finetuning project-specific models down the line.
*Thread Reply:* @Sara Beery Yes, that is something I want to add but it involves quite a bit of work. It's on the todo list 🙂
From the Computational Sustainability network:
CLAIRE Network is collaborating with AIhub.org on a focus series on “AI for Good: UN sustainable development goals”. This series highlights AI research relating to the UN sustainable development goals (SDGs). Each month we focus on a different UN SDG. The first SDG topic, released in January, was SDG: Good health and well-being. You can find the articles published so far collected here: https://aihub.org/tag/focus-on-good-health-and-well-being/ * * The next topic will be SDG: Climate action and they are looking for researchers who work in AI relating to climate action who are interested in writing or talking about their work. You could write (or describe on video) a short blog post about your research, write an opinion piece about an area of current interest, give a tutorial, or take part in an interview. If you are interested, please do get in touch at aihuborg@gmail.com. The deadline for contributions is 12 February 2021.”
*Thread Reply:* thanks. I'd love to stop paying pilots, but yes, a good idea.
*Thread Reply:* i didn't mention, we also fly in a CESNA fixed wing over the everglades.
*Thread Reply:* its old, and a bit scary.
*Thread Reply:* The thing going in our favor is that we are looking at wading bird colonies, which we tend to know their general location. We think (hope) that 90% of the nesting birds are in these known areas. Less spread out than many other wildlife monitoring programs.
*Thread Reply:* @Ben Weinstein what sort of area do you need to cover? What is the mission profile like?
*Thread Reply:* I don’t know Wingtra, but I think you’re on the right general track; fixed-wing drones are really the only option for covering distance. It’s been a while since I looked, but depending on how much you can spend you might look at a UAVFactory Penguin B or C, (https://uavfactory.com/en/penguin-c-uas) which blew away endurance records, or the UAS drones (http://www.ua-sp.com/c-astral) which have much less endurance but are aimed at people needing very precise positioning. If you can afford to spend as much as a modest house would cost, there are even better options :)
*Thread Reply:* @John Payne what sort of endurance / speed / payload do those have? #too-lazy-to-read-link
*Thread Reply:* For the penguin B: 10 kg, 25-50 hours(!), cruise speed of 43 knots, wide variety of payloads available including gyro-stabilized. Astral have only about a 3 hour range, but lots of available cameras including serious multispectral stuff and LIDAR.
*Thread Reply:* I’ve asked UAVFactory for the cost of a basic unit. Those sorts of speed/endurances mean we could cover 800km of flight lines (transects) in a day, which is meaningful. Would still be nearly impossible to get clearances for a large drone here in Tanzania but I’m keen to see what the possibilities are.
*Thread Reply:* … aaand UAVFactory is so far quite elusive about what their systems actually cost …
*Thread Reply:* From what I saw it’s around $15K to $20K for a basic system, but obviously depends on the extremely varied set of options chosen; esp. camera systems — I’ll be interested to hear what you find out.
*Thread Reply:* one thing we are really worried about is wind sensitivity for the fixed wings.
*Thread Reply:* My tech is convinced that wingtra drones will be too shaky in our typical 15mph wind.
*Thread Reply:* Everglades is basically windy 100 days a year
*Thread Reply:* so i contact https://www.atmosuav.com/ and they advertised a heavier drone for windier conditions.
*Thread Reply:* but its more like 25,000$
*Thread Reply:* which is probably just outside our price range.
*Thread Reply:* Yes I think there are two issues: 1) tipping side-to-side makes the camera take photos that aren’t pointed straight down, which could violate your transect assumptions and/or add distortion at the edges of the images, and 2) shaking can cause blurring. Gyroscopes can help with blurring (and tipping to some degree), and gimbals can help with the tipping issue. But both also increase payload. I would think that the lighter weight the camera, the lighter the gyro/gimbal needed to control it, so it might be worth looking at integrated camera/gyro/gimbal systems that have been designed with light weight in mind. UAVFactory makes some (https://uavfactory.com/en/stabilized-payloads), but i haven’t checked them out and I don’t know what the prices are. (Howard, I notice they are ITAR-free). In general, I would think that a UAV with less lift would be less tippy, so a bullet-like UAV might be more stable than a glider-like UAV. I don’t know whether swept-wing designs like the C-Astral drone I pointed to or Marlyn are more stable than straight-wing designs in general — maybe Howard knows?
*Thread Reply:* UAVFactory’s responses (I’ve had 3 sets of back-and-forth so far with no straight answers) were that my (fantasy) budget of $100K that I put down on the contact form was “far too little” for the base model.
*Thread Reply:* is it sensitive to the wind! can you send any sample images?
*Thread Reply:* my techs are convinced it won't work in the wind.
*Thread Reply:* our tech says that, while Palmyra is a very windy place, the Wingtra handles steady winds up to 35MPH. Imagery is good, but suggest shifting to max shutter speed. Landing in high wind is risky. Very gusty winds will require a larger area for takeoff & landing. In general, his impression is that the Wingtra is much more capable in windy situations than a quadcopter.
I can look into a file share if that is still of interest.
*Thread Reply:* If your tech is willing to talk by zoom/phone, I would love to hear.
*Thread Reply:* i'll move this to a DM
New dataset on lila.science, courtesy of the Wild Nature Institute (and lots of helpful volunteers at Zooniverse):
http://lila.science/datasets/wni-giraffes
This dataset contains keypoints for giraffe photogrammetry, so automating this is a potentially interesting ML problem that steps slightly outside the usual detection/classification paradigms, and WNI already makes extensive use of ML tools to accelerate their workflows, so if a student is looking for a nice self-contained ML project that could be used right away by a conservation organization, definitely take a look at this dataset!
Background on WNI's previous work integrating an AI pipeline for giraffe cropping and individual ID:
https://www.sciencedirect.com/science/article/abs/pii/S1574954118300426?via%3Dihub
*Thread Reply:* I can help here too. I know of most of the datasets, ping me if I don't update here during the week. https://github.com/weecology/NeonTreeEvaluation https://www.newfor.net/download-newfor-single-tree-detection-benchmark-dataset/
*Thread Reply:* https://www.planet.com/nicfi/
*Thread Reply:* I'm currently curating a bunch of urban forest data, happy to add once it's ok'd for release!
*Thread Reply:* I'm currently working on a project to use the NICFI dataset to monitor land use in the tropics. We're hoping to use transfer learning and looking for a model trained on 5 layer surface reflectance images (best resolution in the NICFI dataset). I'd be happy to share what we've found so far/bounce ideas off each other if you end up doing something similar.
*Thread Reply:* @Daniel Grzenda love to hear more, what measure are you trying to predict. I have access to the data, but don't see a massive use case yet.
*Thread Reply:* We're working on labeling a subset of the quads with segmentation masks of different industries (mines, mills, plantations, etc) and the plan is to use dice as our metric
*Thread Reply:* Great question, I am also interested. I recently found yolo3 worked quite well in comparison with faster-rcnn type architecture.
*Thread Reply:* The reason was that I wanted to work with the object centric features provided by two stage object proposal then object classification structure of Faster R-CNN. I think there's definitely a lot of interest in a lighter-weight version though! Adapting it in an object-centric way would mean thinking about how to appropriately extract object specific features to compare to, and where in the network you would want to place the attention block. I think there's even possibly a classification version of this where you add attention across entire frames before the final network layers.
*Thread Reply:* Great. Thanks for explaining this Sara!
*Thread Reply:* Haha omg, I'm sorry, I just realized I answered a question you didn't ask! I just answered "Why did you start from Faster R-CNN for Context R-CNN"
*Thread Reply:* The answer to your actual question about the MegaDetector was that on the data we were evaluating on, at the time we trained the model, Faster R-CNN was the highest performing architecture (and did outperform YOLO). I think possibly because we're training on lots of data from different parts of the world the higher capacity model was able to capture more information. However, there have been a couple big advancements in object detection since then (2018!) and @Dan Morris and I were just talking a couple weeks ago about doing a new architecture comparison for MegaDetector V5, and including some of the newer, high-performing models such as RetinaNet, CenterNet, EfficientDet, etc. as well as re-evaluating on SSD/YOLO.
*Thread Reply:* Hi Olga! We all definitely need more help in front end development. Welcome.
*Thread Reply:* They must be doing some NLP here, https://www.zooniverse.org/organizations/md68135/notes-from-nature I know there is transcription.
*Thread Reply:* working on a similar thing now.. paper under review with minor revisions.. will post here soon 🙂
*Thread Reply:* Snapshot serengeti has kinda messy crowdsourced sequence-level behavior labels, but they aren't matched to individuals
*Thread Reply:* I've seen a work recently on tracking and/or estimating pose of individual chimps (https://gdude.de/densepose-evolution/, https://advances.sciencemag.org/content/5/9/eaaw0736?utmcampaign=The%20Batch&utmsource=hsemail&utmmedium=email&hsenc=p2ANqtz-8CwLclbKbf40u8wWvIUWHI963s-JBWbyXpTKSgk7bmicYo17It7L7cubxMB7g0nXbin1fw|https://advances.sciencemag.org/content/5/9/eaaw0736?utmcampaign=The%20Batch&utm_source[…]8wWvIUWHI963s-JBWbyXpTKSgk7bmicYo17It7L7cubxMB7g0nXbin1fw)
*Thread Reply:* But I'm not sure if those datasets are 1) public, 2) have behavior labels
*Thread Reply:* https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0158748
Hey all. We have struggled at Wild Me with finding small batch data annotation outsourcing options, either through volunteers, paid local contractors, or independent firms. Local contractors and volunteers have been problematic, rarely completing one project or lasting more than two. Given that our average batch size is 2000-2500 photos needing annotations, we're not at the volume or price point that a big vendor (iMerit, Samasource, etc.) is interested. We recently started working with a past collaborator (Dr. Richard Lamprey) and his Uganda-based aerial survey analytics team (https://www.wildspace-image-analytics.com/). We have completed three projects with them and are wrapping up a fourth. Their speed, quality and attention to detail has been impressive (the team is composed of experienced aerial survey reviewers), and the price point is good too. I highly recommend them if you are looking for an outsourcing option.
Full disclosure: I am a past collaborator with Dr. Richard Lamprey. I have no financial interest in Wildspace.
More info here as well - https://www.wildlabs.net/tech4wildlife
*Thread Reply:* I build production ready TF apps, but not for the deployment at "consumers/non tech people". I have control about the devices that TF GPU and CPU containers get deployed to or can define requirements about their environment.
To my knowledge the gpu driver, cuda, cudnn version hassle is still present in nvidia-docker. Specially when using new GPUs like the 3060, ... compatibility with every prebuild gpu TF containers is a real pain.
But one of the main benefits is, that you have way more control about the software stack with nvidia-docker, but the pain of supporting older and newer gpus is still present.
Further more I am not sure about the EULA, ... when distributing images publicly.
Side note: If you want to use gpus within docker-compose ping be for workarounds, since its not officially supported yet ;)
*Thread Reply:* If you decide to go with docker images, i could share some of our internal guides privately, that might help you to set everything up and have some orientations for the README.md.
*Thread Reply:* @Frederic Thanks -- I'm glad you've shared this knowledge with me before I started any Docker development. It looks like there may not be much to gain in my use case.
I've thought about just writing a script that downloads and installs all the requirements but NVIDIA don't make it easy (you need an account to download the cudnn libs 😠 ); and I doubt I can just bundle them into my installer without facing licensing issues. Will ping you if I decided to explore things further, thanks for the offer 👍
*Thread Reply:* @Petar Gyurov what do you have in mind? I took a meeting with an engineer not so long ago on, on device camera traps, may be relevant here.
*Thread Reply:* @Ben WeinsteinEnabling GPU support in my project, MegaDetector-GUI, requires end users to download all the CUDA drivers, tookits and library patches. I've written a basic How-To guide but I was kind of hoping for a more elegant "automagical" solution that non-tech people can easily do.
*Thread Reply:* What's the latest in bird sound ID? What's new? Are we getting better? What's the main obstacle, annotations? geographic generalization? data quality?
*Thread Reply:* My perception (keeping in mind I’ve only been working on this for 6 months) is that there is a lot of room for improvement in getting more annotated data. There exists a ton of high quality recordings (eg at Macauley library), but it’s a bit of an expert task to box and correctly label each bird vocalization.
*Thread Reply:* Species recognition in soundscapes remains hard, because there tends to be many overlapping sounds, birds at varying distances from the microphone, and lots of background noise.
*Thread Reply:* I think most existing approaches to bird sound recognition use tweaks of off-the-shelf vision models (eg ResNets), and it would be exciting to construct some models which are more custom-built for the domain.
*Thread Reply:* interesting. so its in the localization of the annotations within samples. I've heard this before. Thanks for your thoughts.
*Thread Reply:* no problem, i appreciate the interest!
*Thread Reply:* It's always interesting to see how many common threads there are across domains!!
New python API for GBIF data, focused on pulling data for ML training!!! If you're interested in testing or contributing you can contact gbif-dl@inria.fr
"We released a first version of GBIF-DL (GitHub, pypi) a package that makes it simpler to obtain training images from the GBIF database to be used for training machine learning classification tasks. It wraps the GBIF API and supports directly querying the api to obtain and download a list of urls efficiently (based on asyncio) and with many options (balancing, bounding, filtering, etc.). We are looking for some potential users who would be willing to test it, provide feedbacks and/or maybe contribute to it."
*Thread Reply:* this is so cool -- thanks for sharing! being a bit of a gbif fanboy myself, this has me pretty excited.
*Thread Reply:* This may come in handy as we start to expand BearID to more species! Is there a way to filter for images that actually show the species in question rather than signs of the species (tracks, scat, etc.)?
*Thread Reply:* @Ben Weinstein yes - have been doing this for a few years with nadir and oblique images from Cessna 182/206.
*Thread Reply:* One of the systems is documented here: https://github.com/TZCRC/Lanner-CamPod
*Thread Reply:* You mention “wiring the camera to a GPU” meaning you’d like to do in-flight processing to discard images? I’d suggest that the cost of storage is very cheap but the cost of missing data by poorly-filtered in-flight processing is high, so think about post-processing. We take images at 2 second intervals - Nikon raw @ 30+MB per image over 4 hours = 200 GB per flight; there are 256 and 512GB cards, but the biggest limitation is speed of the SD card (if it isn’t fast enough you get gaps in coverage while the camera catches up with writes).
*Thread Reply:* Is someone physically changing out the SD cards as they fill during flight? My vision for the inflight processing is just have a pilot flying. That way we have a TB drive in the plane, but I guess we can just skip the detection mid-flight. Especially due to COVID, we can only have one person in the plane.
*Thread Reply:* No, it's a single SD card per flight. We were fine with 256GB until now but upgrading to 512 soon for longer flights. You planning on raw files?
*Thread Reply:* The reliability of a Nikon to get a photo on the sd card with every triggering is rock-solid. There are probably ways to record to an external drive or computer but don't lose the onboard reliability.
*Thread Reply:* Great. I'll take this to the team. Thanks.
*Thread Reply:* Hi Ben - are you sure that you need to store RAW files? If you stored high quality JPG you may be able to reduce the total storage just about to where youre fast SD cards will manage for the whole flight and then process once you are on the ground.
*Thread Reply:* @Ben Weinstein Would you be willing to share your image dataset with the students in our class? As I understand it you need a good algorithm for detecting and counting the birds, where each bird is assigned a latitude and longitude. You probably want to also estimate the species. Correct?
*Thread Reply:* @Ben Weinstein If you are inclined to share (part of) your dataset with us I will be in touch to work out the details.
*Thread Reply:* Hi, Vincent. We are developing a platform for capture, management and administration of biodiversity collections at our group in CONABIO and we are using/testing it internally. Our intention is that any research group could use it, but for now it is still in developing phase 😔 maybe later! 👋:skintone4:
*Thread Reply:* Bonjour Vincent! Do you have camera trap datasets and image analysis challenges that you are willing to share with our class at Caltech?
*Thread Reply:* @Vincent Miele CNRS We (@Sara Beery, @Elijah Cole (Deactivated) and Neehar Kondapaneni) are teaching a computer vision class at Caltech and would love to challenge our students with projects that help ecologists and field biologists.
*Thread Reply:* Bonjour Pietro, we are discussing with different french partners to get image data, but these are their own private data... Maybe they will share their data at some point in the mid-term, but this is not planned at the moment. In any case, this would be a good idea!
*Thread Reply:* what does underwater camera trapping look like? Can you post a photo? My old tool for video processing had some object detection that seem to get alot of downloads from the underwater ecology community. We are redesigning those tools, but don't have a well annotated underwater dataset.
*Thread Reply:* @Elizabeth Madin welcome … and do you have any writeups on underwater camera traps? Am trying to figure out how to make some for cichlid monitoring in freshwater and am struggling!
*Thread Reply:* thirding @Ben Weinstein and @Howard L Frederick here - I would love to see you underwater camera set up! I’ve been building species ID models with the collected habcam data along the NE coast of the Gulf of Mexico that look like this and I’d love to know if folks have come up with resilient cages for their cameras that don’t have bars…
*Thread Reply:* This is the preliminary results of species ID detection (trained with mmlab’s toolbox & resnet50 backbone), and I bet consistent accuracy would increase without these bars…
*Thread Reply:* That looks so cool, @Océane! Thanks for sharing. ♥️
*Thread Reply:* @Océane that is a very solid looking camera, do you have any public build guides for how you did it?
*Thread Reply:* @Ben Weinstein and @Howard L Frederick: this paper describes in some detail what we did recently with our underwater (coral reef) camera trap arrays. I call them camera traps - it's actually continuous video that we manually annotated (ugh!). We have a fairly large dataset (~3000 annotated individuals, mostly fish + some inverts) from Australia's Great Barrier Reef that could be put towards developing an algorithm if people are looking for a challenge like that! We're collecting and annotating more video now from Hawaii (some species overlap with Australia, but not complete) and would absolutely love to apply an algorithm like what you've posted, @Océane! I'll also post a photo from our earlier rounds (using now-old GoPros) - we don't bother with cages because we've never had a problem with cameras being dislodged, attacked, etc., We're working in fairly wave-limited coral reef lagoon environments.
*Thread Reply:* That little camera is so cute! Is it mounted directly on the coral/rock?
*Thread Reply:* @Océane: It's just temporarily cable-tied to dead coral. Easy on, easy off with no damage to anything. 🙂
*Thread Reply:* @Elizabeth Madin interesting paper, it’s amazing to be able to quantify feeding behaviour like that. I notice 7 cameras malfunctioned (out of 30?) - what were the problems there?
*Thread Reply:* @Howard L Frederick: Ah, yes...we had signifiant camera failure because we had to modify the cameras slightly by using a larger waterproof housing case to accommodate the extended-life batteries we needed to get us through dusk and into the night. (We had to set out the cameras and get the boat back to the research station prior to dark.) A number of the modified housings failed overnight, despite having passed our (shorter-term) tests at depth prior to deployment. It was indeed pretty frustrating, but thankfully we still got enough data from the remaining cameras to get to the answer!
*Thread Reply:* @Océane, @Elizabeth Madin, from ecolog if you didn't see it
Hello, my name is Austin Greene and I am a PhD student at the University of Hawaii at Manoa studying the ecology of coral reefs. I recently developed a low-cost camera system (KiloCam) that I hope will make habitat monitoring more accessible, and I am looking for other researchers to help me field test it. KiloCam is small, about 40x25x25mm, and fits inside of cheap GoPro housings to be made waterproof. The instrument is capable of taking 2 MP photos at user-specified intervals, can run off of many power sources, and is highly efficient for long deployment times. For example, on a set of two AA batteries KiloCam should operate for over a week at one photo per minute, or nearly a year with one photo per hour. While the camera isn't particularly high-resolution and is best suited for environments with ample ambient light, it is adequate for habitat monitoring. My hope is that the low cost of building KiloCam (~$30 USD) will make mass deployments possible, or allow improved habitat monitoring in underserved areas.
*Thread Reply:* Hello! We have an earth observation dataset at MIT, of satellite images from Sentinel. It's about 100k pairs of images from the arctic in the winter and the summer. We are currently fixing how diverse the dataset is, so it could get smaller. But, please let us know if you want to use it for the class.
*Thread Reply:* @aruna Sounds like it could be a good fit! What’s the task / label set like?
*Thread Reply:* We have winter tiles, (pre-melt), summer tiles (post-melt), and segmentation masks that identify ice and non-ice regions.
*Thread Reply:* @aruna that sounds like a great dataset. Is it public? Or if not, would you be able to share it?
*Thread Reply:* Yes, I can share it with you. We are currently working on paring it down a little to have more diversity in the tiles, but I can send you the details once it is published.
*Thread Reply:* Can you give me a date I should aim for to get the dataset to you?
*Thread Reply:* The class will begin at the end of the month!
*Thread Reply:* Sounds great. Will send it to you folks soon.
*Thread Reply:* Just sharing a few things that might be fun starting with the same data. I can expand on any of these ideas.
*Thread Reply:* 1. Object detection from multiple sensors with uncertainty in spatial alignment. https://github.com/weecology/NeonTreeEvaluation
*Thread Reply:* 2. Fusing 2D sensor and 3D point clouds in object detection. https://github.com/weecology/NeonTreeEvaluation
*Thread Reply:* 3. Geographic generalization in fine-grained species classification https://github.com/Weecology/DeepTreeAttention
*Thread Reply:* 4. 3D CNN versus stacked 2D CNN versus apriori feature engineering for hyperspectral deep learning (369 bands). https://github.com/Weecology/DeepTreeAttention
*Thread Reply:* 5. Long tailed species classification and unknown class identification. https://github.com/Weecology/DeepTreeAttention
*Thread Reply:* 6. Miniaturization of object detection networks for in-flight screening of remote sensing imagery during wildlife surveys http://tree.westus.cloudapp.azure.com/everglades/
*Thread Reply:* Hey @Ben Weinstein if we get a student team or two interested in these would you be able to help mentor them?
*Thread Reply:* sure, I can't promise a ton of time, but alittle
*Thread Reply:* Then there is the hummingbird data. Key frame detection and tracking in low-frame rate videos. https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13011
*Thread Reply:* Some of these are more ready for students then the others. #6 and #2 are pretty plug and play.
*Thread Reply:* also we ran a competition last year that covers the general (easier) tree species classification and bounding box detection. The docs and data are all there.
*Thread Reply:* https://idtrees.org/competition/
*Thread Reply:* https://zenodo.org/record/3700197#.YEVK7pNKiHE
*Thread Reply:* @Ben Weinstein Ben - thank you very much for all the suggestions!!
*Thread Reply:* We are currently running a competition on high-resolution land cover change detection from weak low-resolution labels (https://www.grss-ieee.org/community/technical-committees/2021-ieee-grss-data-fusion-contest-track-msd/). The competition will be over next month (it is currently in the closed test phase), but the evaluation server/data will remain up.
*Thread Reply:* There is an interesting super-resolution component from the CV side (and of course identifying land cover change over time has strong ecological implications!)
*Thread Reply:* Many thanks to @Caleb Robinson @Ben Weinstein @aruna
*Thread Reply:* It would be great if you could also record and/or post the lectures somewhere! I would love to watch them, and I'm sure others would too.
*Thread Reply:* We're planning on hosting it all publicly 🙂
*Thread Reply:* A bunch of us have been working on detecting tree swallow roosts from RADAR data. We are planning to release a standardized benchmark around this. Would be happy to share it. What’s the timeline for the course? @Dan Sheldon and I would could also do some lightweight mentoring if there’s interest on working on this. More info: https://people.cs.umass.edu/~zezhoucheng/roosts/
*Thread Reply:* The class will be starting end of this month, so the timeline is a bit tight, but I think that would be an amazing application to bring in and we would be incredibly happy to have you act as high-level mentors. Let us know if that timeline might be possible?
*Thread Reply:* Another option might be to bring in an UG team to help with testing and publishing the benchmark as part of their project? Different members could run different baselines and provide analysis of efficacy
*Thread Reply:* Awesome! I think it’s doable --- we are already working on this and the class deadline is a good motivation 🙂
*Thread Reply:* 👋 @Stefan Schneider! Have you looked at requesting tiles from Pleiades, offered by the ESA?
*Thread Reply:* It's free for research, and you can get all areas covered by sea. Please take a look at their area coverage, since I am not entirely sure if they photograph land and water bodies both. It has excellent resolution at 50 cm/px
*Thread Reply:* @Stefan Schneider, have you met @Patrick Gray, who did this work while working for our old boss Ari Friedlaender. I can put you in touch with Ari as well. Also check with the NOAA authors who have this data. I also know of a talk by people at Vulcan Inc, interested in this as well. Being in canada, you probably talked to https://www.whaleseeker.com/about-us. It feels like someone needs to organize all these folks.
*Thread Reply:* Hi @Stefan Schneider! I don't have any satellite imagery I can share - Maxar is typically protective of their licenses - but we trained our satellite whale model primarily using down-sampled aerial imagery, which is available here: https://github.com/lynch-lab/Borowicz_etal_Spacewhale
*Thread Reply:* If I remember right, Guirado et al. used Google Earth images - doable, but you lose a lot of control because of Google's pansharpening algorithm. They're also not full resolution and it's not super clear how much that matters if you want to feed in full-res imagery
*Thread Reply:* Hi Aruna. Thanks for this! I'll definitely take a look through the ESA tiles from Pleiadas
*Thread Reply:* @Ben Weinstein It would be amazing if you could put me in touch with Patrick Grey & Ari Friedlaender! Which NOAA authors are you referring to (sorry if I missed something)? Vulcan Inc sounds like a great contact. Same with this Montreal group that I'll definitely reach out to. Thanks so much!
*Thread Reply:* @Patrick Gray should be wandering around somewhere. If doesn't jump in, i'll email him.
*Thread Reply:* @Alex Borowicz This is great! I'll definitely take a look at the repo/data and let you know if I have any questions. Cheers!
*Thread Reply:* Hi all - The Nature Conservancy is scoping the potential for use of VHR satellite imagery as a tool to survey large whales to meet conservation monitoring objectives in California, so it's great to see mutual interest in this topic. @Stefan Schneider we'd love to hear more about you work in Canada and see if there are ways to collaborate.
*Thread Reply:* Hey @Stefan Schneider! I know some of your papers pretty well, good to hear from you. My advisor and some collaborators are still working on this though I've moved on mostly. They're taking a large archive we have of drone imagery of whales and downsampling to satellite imagery resolution and using that to train a model for right whales. Working with some folks from BAS and Canada's DFO. When I was doing this I was using all WorldView data and it was working alright but a huge effort to find suitable training data as you likely well know. I ended up largely just searching in common bays and breeding grounds to build up an initial dataset and then iteratively applied the model to find more. Happy to answer any specifics but again I'm a little out of date now on this work!
*Thread Reply:* @Vienna Saccomanno Great to make the connection! It's early days that are just ramping up, but I'm excited to connect with others and hopefully collaborate. Right now it's the pursuit of initial datasets to build a base line of what we can expect from an object detector.
*Thread Reply:* @Patrick Gray Thanks for reaching out! It's great to chat with someone who has tackled this problem. Down sampling from drone imagery sounds like a nice solution. Most definitely with the suitable training data. I'm more or less surveying right now trying to find any sort of public dataset. Would you happen to have a link the WorldView data you were using? Or anything that may act as a thread to pull? Cheers!
*Thread Reply:* Hey @Stefan Schneider unfortunately this was all from access through a NSF polar program grant (via the Polar Geospatial Center) and we can't share any of the data. I've also since lost access myself so can't pull any more. Maxar doesn't seem as interested in this problem as they did in their Digital Globe incarnation.
*Thread Reply:* @Patrick Gray no worries. This seems to be the way it goes for valuable datasets. I'm sure we all wish we could bring all this data together to train a universal performant model
Are there folks who have used https://github.com/knjcode/imgdupes?
*Thread Reply:* Ah, it's related to the hamming distance. 🙂 All good here.
*Thread Reply:* I believe most of them are on arxiv? Is there one you can't find?
*Thread Reply:* It looks like there are direct links here as well: https://sites.google.com/corp/view/fgvc7/program?authuser=0
*Thread Reply:* Exactly what I was after. Thank you!
*Thread Reply:* Hi Ben, I am not an expert in this area, but I am taking a computer vision class this semester, and one approach we spoke of here is to use a gaussian blur + sub-sampling. It's one of the recommended ways to downscale images, and between successive down-sampling, you lose mostly details. Plus up-sampling with Laplacian pyramids. Sorry if you already knew this, was excited about this class from today morning, and jumped in with a response. : )
*Thread Reply:* I remember @Sarra Alqahtani discussing challenges surrounding switching between actual data collected with different resolutions vs subsampling high resolution data in her mine detection work?
*Thread Reply:* Hi @Ben Weinstein. Sorry for the late response. What we are currently doing is to train our model on a mixed of high resolution images (from drone) and downsampled images (low resolution from satellite) then during testing we only use low resolution images. We use GANs and it is a risky technique since the model may populate unreal details but our point is to improve the accuracy of the object detection so we are not that worried about the accuracy of the resolved images (landscapes with gold mining objects). I would think your goal is the same unless you want to differentiate between birds. I hope this helps.
*Thread Reply:* have you seen this strategy elsewhere? When you say a mix, do you mean at the same time, or sequentially?
*Thread Reply:* it is like 80% of the dataset is hr and the rest for the lr. I will find the papers we got this idea from. You could try the sequential and see if it will improve the results just as a form of TL
*Thread Reply:* awesome. We have alot of use cases for cross resolution work, and never really explored the literature. Thanks.
*Thread Reply:* @Sam Kelly might have a good sense for the right edge device?
*Thread Reply:* I am personally a fan of Google Coral ecosystem - very easy to use. As with any edge device, figuring out a way to keep the power down is the key! (This is something we have been working on at CXL) I would be more than happy to jump on a call with you to help out. just DM me
*Thread Reply:* Thank you! That would be great, we will message
*Thread Reply:* TrailGuard AI project has used an Intel’s movidius myriad 2 VPU . Edge devices, you can also test: nvidia jetson nano. I made a camera trap with the Google Coral devboard.
iWildCam 2021 is LIVE!!! https://www.kaggle.com/c/iwildcam2021-fgvc8
"Camera traps enable the automatic collection of large quantities of image data. Ecologists all over the world use camera traps to monitor biodiversity and population density of animal species. In order to estimate the abundance and density of species in camera trap data, ecologists need to know not just which species were seen, but also how many of each species were seen. However, because images are taken in motion-triggered bursts to increase the likelihood of capturing the animal(s) of interest, object detection alone is not sufficient as it could lead to over or undercounting. For example, if you get 3 images taken at one frame per second and in the first image you see 3 gazelles, in the second you see 5 gazelles, and in the last you see 4 gazelles, how many total gazelles have you seen? This is more challenging than strictly detecting and categorizing species as it requires reasoning and tracking of individuals across sparse temporal samples."
Please share to your communities, and if you're on twitter help me out by sharing our twitter thread! https://twitter.com/sarameghanbeery/status/1370065404148678657
Similarly, the iNat Competition is back for 2021!
https://www.kaggle.com/c/inaturalist-2021/overview https://github.com/visipedia/inat_comp/tree/master/2021
```After taking a year out, we are excited to announce the latest iNaturalist Species Classification Challenge. This challenge is part of the Eighth Workshop on Fine-Grained Visual Categorization (FGVC8) at CVPR 2021.
The iNat Challenge 2021 dataset contains 10,000 species, with a training dataset of 2.7M images that have been collected and verified by multiple users from iNaturalist. There is also a more manageable "mini" dataset with 50 images per species, for a total of 500K training images. The dataset features many visually similar species, captured in a wide variety of situations, from all over the world.
We have made several modifications to the competition this year. Similar to the 2017 competition, we are releasing the species names immediately, instead of obfuscating them. Our reason for obfuscating them in 2018 and 2019 was to make it difficult for competitors to scrape the web (or iNaturalist itself) for additional images. Because we are releasing 2.7M training images and the dataset doesn't necessarily focus on the long tail problem we feel that we can release the species names without worry. This does not mean that scraping is allowed. Please do not scrape for additional data, especially from iNaturalist or GBIF. Having the species names also makes interpreting validation results easier when examining confusion matrices and accuracy statistics.
We are also releasing location and date information for each image in the form of latitude, longitude, location_uncertainty, and date values. We have retroactively added this information to the 2017 and 2018 datasets, but this year competitors are able to utilize this information when building models. We hope this motivates competitors to devise interesting solutions to this large scale problem. You will find more information on the challenge Github page.
We are looking forward to seeing the creative solutions you come up with!``` Please share to your communities! https://twitter.com/oisinmacaodha/status/1370050350779150340
*Thread Reply:* I just saw https://www.frontiersin.org/articles/10.3389/fmars.2021.629485/full
*Thread Reply:* Not sure how many species, so it may not be very fine-grained
*Thread Reply:* And there was a fish task in LifeClef a while ago https://www.imageclef.org/lifeclef/2015/fish
*Thread Reply:* We may have a relevant dataset...will send details when I reply to your email, but feel free to remind me if I forget.
*Thread Reply:* Depending on exactly what you're looking for, here are a few more:
https://github.com/Microsoft/Project_Natick_Analysis/releases/tag/annotated_data
http://www.inf-cv.uni-jena.de/fine_grained_recognition.html#datasets (link is called "Croatian Fish Dataset")
...and, yes, of course there's a dataset called "DeepFish":
https://alzayats.github.io/DeepFish/
...in fact it's a pretty amazing dataset!
New data set on LILA containing ~100k bounding boxes on trees in drone images, including classification of invasive moth damage on a subset of those bounding boxes.
http://lila.science/datasets/forest-damages-larch-casebearer/
Courtesy of the Swedish Forest Agency.
*Thread Reply:* Ooo! Me! I will download and train on this soon
*Thread Reply:* This is what a sample image looks like. If I get a moment I will generate a single csv file so others don't need to parse the xml.
Consider adding EAAMO to your annual rotation of conference deadlines! Your work on AI for conservation (both technical and applied) will be very welcome.
Stemming from the Mechanism Design for Social Good (MD4SG) initiative, the inaugural ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO’21) aims to highlight work where techniques from algorithms, optimization, and mechanism design, along with insights from the social sciences and humanistic studies, can help improve equity and access to opportunity for historically disadvantaged and underserved communities.
Website: https://eaamo.org/ Submission date: June 3 Conference: October 5–9
Great to meet all of you on this channel. My name is Halil Radogoshi and I am a Senior Advisor on knowledge management and AI at the Swedish Forest Agency. We have started several AI projects at the agency and the projects mostly deal with forest damages. Here is a link to a video https://www.youtube.com/watch?v=WcVzys6ECys from one of our projects. Looking forward to exchange experience on this channel. Thank you @Dan Morris for inviting me here.
*Thread Reply:* I can't quite find a list of tasks, was 'is this hummingbird visiting a flower' an example, or an actual task?
*Thread Reply:* I have a few dozen TB of video that throw at that question.
New data set on LILA; ~50k bounding boxes on bees and pollen:
http://lila.science/datasets/boxes-on-bees-and-pollen
...courtesy of the BeeLivingSensor project.
Hi everyone, the Herbarium Challenge 2021 is currently running! 🌸🌿 Still two months to go before the final submission deadline. https://www.kaggle.com/c/herbarium-2021-fgvc8/overview
*Thread Reply:* thanks for sharing here! I posted it on Twitter but forget to share here as well
*Thread Reply:* Hi Carly, do you manage this? Very cool to see all these! Here’s another one: https://www.sharkeye.org/
*Thread Reply:* And https://www.fishnet.ai/
*Thread Reply:* I don’t! I just came across it and wanted to share. I believe it is managed by Kate Wing? kate@katewing.net, https://www.katewing.net/projects
Wow, we are clearly doing a bad job of making Skylight visible. Do you know the owner of the list? This is a good list but I see several things missing. Is this something they want to share and have the “community” add to?
*Thread Reply:* I'm sure they would love additions (or at least I would if I were them!). but I don't know them personally.
Looks like the owner is Kate Wing, maybe you could reach out to her directly? https://www.linkedin.com/in/kate-wing-11b7029
*Thread Reply:* definitely reach out to the folks at Osa Conservation! https://osaconservation.org/ They do a ton of work in the Osa peninsula region, working with communities to conserve forests, wetlands, coasts, and they have a sustainable ag program
*Thread Reply:* I’m currently at La Cotinga in Osa 🙂 but heading towards a large restoration project at the border of Panama, send me a DM!
*Thread Reply:* Did you talk to @Benjamin Kellenberger?
*Thread Reply:* Also I know you chatted with Mark Koneff, but just in case I think there might be several others with datasets in the Community of Practice on migratory bird surveys he's running. Maybe we could email that group?
*Thread Reply:* I talked to FWS this morning, but didn't know there was a group email.
*Thread Reply:* He has a mailing list for everyone who has been involved
*Thread Reply:* Could probably pass on a request if you write up a blurb
*Thread Reply:* Hi! I can answer @Ben Weinstein’s question here: our birds dataset over West Africa is publicly available on lila.science: http://lila.science/datasets/aerial-seabirds-west-africa/ The accompanying paper can be found under this link: 21 000 birds in 4.5 h: efficient large‐scale seabird detection with machine learning - Kellenberger - - Remote Sensing in Ecology and Conservation - Wiley Online Library
*Thread Reply:* my bad! its not listed here http://lila.science/datasets
*Thread Reply:* @Dan Morris or @Siyu Yang, bump to add to front page. Is there another way to look at this list besides the dataset tab. In case I missed anything else.
*Thread Reply:* Those of you who have contributed data to LILA know that we send out a password-protected preview page first; when we make the page live, we have to un-check two boxes (for password and listing), and I forgot to un-check one of the two boxes.
*Thread Reply:* thanks for all this organization work, it is crazy important. I'm rounding up UAV datasets for bird detections. No idea yet is atleast a baseline model is tangible in this area, the problem is pretty unconstrained across taxa, background, spatial resolution, etc.
*Thread Reply:* There were a number of us interested in the same question for monitoring seabirds on islands in the pacific - and started the #ai4pacificislandseabirds channel. @Mari Reeves @Dena @Maddie Hayes @Vienna Saccomanno all have active projects monitoring seabirds. We could also help connect you with other datasets - let me know if you want to connect.
*Thread Reply:* Thanks @David Will, I have data from @Vienna Saccomanno already. I definitely still look for more data, either annotated (preferably) or unannotated that fits the general specs of visual birds in UAV imagery <3cm resolution.
*Thread Reply:* Anyone looked at an approach like this for counting birds? https://arxiv.org/abs/2104.08391
*Thread Reply:* I'm interested in this for boobies in Lehua. Can we talk?
*Thread Reply:* One way we've thought about it is incorporating species distribution models from something like Map of Life. @Elijah Cole (Deactivated) or @Kevin Winner would be able to say more 🙂
*Thread Reply:* Hey @Ben Weinstein! We have had similar thoughts, but the closest we have currently gotten is to use of externally generated range maps for eval (a bit like @Sara Beery's comment). These are still not perfect e.g. if you click on the Map tab halfway down this page, you see that there have been observations in regions that are not inside the range (pink area). https://www.inaturalist.org/taxa/42412-Ovibos-moschatus
*Thread Reply:* Obviously, this is a very anecdotal example, but issues of this type as well as "range creep" do occur:
*Thread Reply:* yeah, its interesting, your problem is incredibly hard because you couldn't have a person write a prior for every grid cell (though a hand drawn map would be cool).
*Thread Reply:* for @Vienna Saccomanno she has islands that are a fixed unit and the species classes occur in somewhat known combinations at each island.
*Thread Reply:* not every island is in train, or else we would just do a site level embedding.
*Thread Reply:* The other question would be, is there a way to do the human in the loop version efficiently for tens of thousands of species. Im also assuming that the common (i.e. easy to guess the range of species) are well represented in your dataset anyway.
*Thread Reply:* so I was thinking about having her parameterize a multinomial distribution by hand.
*Thread Reply:* The island example is a great one.
*Thread Reply:* The current hack would be to just manually generate some "pseudo positive" training examples and add them to the training set.
*Thread Reply:* rather than going to https://botorch.org/?
*Thread Reply:* The challenge with any of this within the current framework (bayesian or not) is that I suspect it will be very difficult to influence the output of the network with a very small number of examples in very localized regions. The network will basically have to devote capacity to remembering just those observations.
*Thread Reply:* I think integrating HSM (vis a vis @Sara Beery’s suggestion) is really the way to go here. If ecologists believe those HSMs actually mean something, they should be useful for narrowing down a species identification.
Call for papers for ICML Climate Change AI Workshop:
"Climate Change AI will be hosting a virtual workshop on climate change and machine learning alongside the ICML 2021 machine learning conference in July. The goal of this workshop is to facilitate networking and exchange between those working in machine learning and those working in energy/climate-related areas.
There are several ways to participate: • Be a mentor or mentee in our submission mentorship program (applications due April 28th) • Submit a paper or proposal on your work at the intersection of climate change and machine learning (submissions due May 31st) • Attend! The workshop will be virtual on July 23rd or 24th, 2021. Thinking about submitting? We are holding webinars to explain what we are looking for on Friday, April 23rd (at 1.30PM Eastern Time / 6:30 PM London time) and Tuesday, April 27th (at 9:00 AM Eastern Time /* *2:00 PM London time / 9:00 PM Beijing Time). We will give advice on how to prepare a successful submission, and an opportunity to ask questions regarding the mentorship program.
For more details, see: https://www.climatechange.ai/events/icml2021.html or reach out to climatechangeai.icml2021@gmail.com with any questions"
thought this might be of interest to folks here who might be curious about combining their research with artistic expression. proposals due by may 2. https://zkm.de/en/open-call-viable-data-remote-residency
*Thread Reply:* Awesome! I love hearing that MegaDetector is useful 🙂
*Thread Reply:* Extremely! And it's accurate too, even for finding creatures in deep, dark Tasmanian rain forests ;)
*Thread Reply:* Our latest version is very good, and we're working on various ways to improve it further. We recently did a test on 22K out-of-sample images. This was the result (top-1):
*Thread Reply:* Wow, that's fantastic! Those numbers look great 🙂
*Thread Reply:* Just a better format, sorry for the replacement.
*Thread Reply:* Yes, we're very happy with it's performance!
*Thread Reply:* Super impressive for the Bassian thrush with 27 examples
*Thread Reply:* It's currently trained on ~300K expert-labelled images, but next round will be on >500K, with some new categories added, and higher version of EfficientNet (the above is B3).
*Thread Reply:* Oh, remember, that was just for the test set. It was trained on about 2K Bassian images.
*Thread Reply:* ahhhhh gotcha, I thought that was training n
*Thread Reply:* It is, given that Bassian Thrushes are actually super camouflage experts!
*Thread Reply:* How do you use MegaDetector in combo with this model? Are you training on crops? Or is it more of an ensemble approach?
*Thread Reply:* Exactly. We take the MegaDetector Crops, and resize them to 256px for EfficientNet. Then we write the species-label to an unused metadata field on the image, and upload to Camelot for final human verification.
*Thread Reply:* Oh, very cool that your workflow integrates with Camelot too!
*Thread Reply:* Yep, it's an essential part of the pipeline. We then analyse the data by importing the Camelot database (full export) in to R, for ecological metrics etc.
*Thread Reply:* With the images 'pre-classified' by our EfficientNet model, the human labelling of images in Camelot is about 10-50 times faster than if we were presented with the usual mixed bag one would get from a direct import of images.
*Thread Reply:* I love that you have human verification built in to your pipeline, and that the species labels help with speedup. Have you thought about how to make humans more efficient? So far I still feel like we need to have human eyes on the IDs before using them to help mitigate ML biases, but it's hard to scale.
*Thread Reply:* I've been thinking about ways to prioritize corrections/verification or potentially use a subset as quality control to estimate error, but everything is so imbalanced that it's hard to do robustly
*Thread Reply:* Well, with the Camelot interface, a human is presented with 150 thumbnails on the left sidebar, which on a large screen still show quite a bit of detail. These can then be bulk-selected, and then the human simply has to look for anomalies within the 150, and deselect those. This is what makes it so fast for us.
*Thread Reply:* That's awesome. Have you done any analysis of whether this causes any bias against small animals?
*Thread Reply:* This is an example search for class 3 (Tasmanian devils).
*Thread Reply:* Re: my above comment, just thinking about thumbnails
*Thread Reply:* and also, do humans verify the empties too?
*Thread Reply:* It might be that some small mammals are overlooked, yes, although the detector does a good job at finding/boxing them in general, and the boxes are all quite visible on the thumbnails.
*Thread Reply:* Yes, all the empties are checked.
*Thread Reply:* that's rad. Do you have any stats on corrections? How many missed animals you see?
*Thread Reply:* After we run the detection over our images, we partition each camera into “animal”, “human” and “blank” folders. I then manually scan the blank folder, and pick up any of the rejections and move them over to the animals folder. It doesn’t take too long to do this (typically an hour or two on a 25K image service), and it avoids having lots of false detections in the final upload to Camelot (where they would otherwise have to be rejected by a person anyway). So it’s a trade-off as to where we want the effort focused --- I like to have a clean dataset going into the database!
*Thread Reply:* in the last set of images we processed, we set its confidence threshold to 95%. For 23,894 new images, it failed to detect 1,054 real images (4.7%) and falsely detected 331 (1.4%). Pretty good!
*Thread Reply:* Thanks for answering so many questions!!
*Thread Reply:* My pleasure -- thanks for all your work on this Sara!
*Thread Reply:* We were talking in our group about ways the detector could be further improved, with a view to helping classifiers. The ideal would be if a future ‘bounding box’ was not a rectangle, but a complex polygonal outline (i.e., tracing, even roughly, the animal profile). This would mean the image information about the animal would be extracted with as little ‘noise-injecting’ background as possible (even with tight boxes, often up to a half of the pixels is still non-animal). But I acknowledge that such an approach might be quite difficult/tedious to train!
*Thread Reply:* Sooooo, if you want to try something fun, test your data on this demo: https://github.com/tensorflow/models/blob/master/research/object_detection/colab_tutorials/deepmac_colab.ipynb
*Thread Reply:* It does class-agnostic segmentation if you provide boxes, and I've found it works astonishingly well on my CT data
*Thread Reply:* Could be fun to use it as weak attention for a classifier, or use this as a rough way to build the polygon labels you mentioned without needing to explicitly collect a bunch of them
New data set on LILA, courtesy of NOAA Fisheries:
http://lila.science/datasets/noaa-arctic-seals-2019/
~28k bounding boxes on seals in ~80k aerial images (color + IR).
Not to be confused with the previous big data set of seals and bounding boxes on LILA. This newer data set represents totally new images from a more recent survey, and a substantial improvement in both the precision of the annotations and the precision of the color/IR alignment.
Shameless plug, but there are so few jobs in this field that I figure it's also a public service announcement: The EarthRanger team has three open job positions in Seattle for people who are passionate about conservation and conservation technology: • UI/UX Designer • Mobile Software Development Lead • Software QA Lead If you know of anyone interested, please let me know and/or direct them to https://vulcan.com/Careers.aspx . Thanks!
*Thread Reply:* @Maxime Vidal has been looking at Re-ID recently
*Thread Reply:* Perhaps also worth posting on Wildlabs in the AI for conservation group?
*Thread Reply:* @Carly Batist that's a good idea
*Thread Reply:* @Utkarsh Goel from the OOCAM team may be able to help
Potential event of interest - https://www.meetup.com/DataForGood-CorrelAid-X-Netherlands/events/277277935/
Pretty awesome paper:
Priority list of biodiversity metrics to observe from space
Monitoring global biodiversity from space through remotely sensing geospatial patterns has high potential to add to our knowledge acquired by field observation. Although a framework of essential biodiversity variables (EBVs) is emerging for monitoring biodiversity, its poor alignment with remote sensing products hinders interpolation between field observations. This study compiles a comprehensive, prioritized list of remote sensing biodiversity products that can further improve the monitoring of geospatial biodiversity patterns, enhancing the EBV framework and its applicability. The ecosystem structure and ecosystem function EBV classes, which capture the biological effects of disturbance as well as habitat structure, are shown by an expert review process to be the most relevant, feasible, accurate and mature for direct monitoring of biodiversity from satellites. Biodiversity products that require satellite remote sensing of a finer resolution that is still under development are given lower priority (for example, for the EBV class species traits). Some EBVs are not directly measurable by remote sensing from space, specifically the EBV class genetic composition. Linking remote sensing products to EBVs will accelerate product generation, improving reporting on the state of biodiversity from local to global scales.
https://www.nature.com/articles/s41559-021-01451-x
*Thread Reply:* This sounds rad - anyone have a pdf version they can share?
*Thread Reply:* Hi there. We will be sending out instructions very soon.
*Thread Reply:* Thank you @Oisin Mac Aodha! Response so speedy. much fast.
*Thread Reply:* PS the workshop is on June 25th.
*Thread Reply:* Thanks! That gives me a window to plan around in.
*Thread Reply:* No, I don't believe that exists. My tree model does an adequate, but not great job, you could retrain it. ```from deepforest import main from deepforest import visualize from matplotlib import pyplot as plt from skimage import io
m = main.deepforest() m.use_release() img = io.imread("/Users/benweinstein/Downloads/image.png")
Remove sneaky 4th transparent alpha channel
m.config["scorethresh"] = 0.01 boxes = m.predictimage(img[:,:,:3].astype("float32"), returnplot=False) boxes.label = 0 image = visualize.plotpredictions(img, boxes, color = (255,0,255)) plt.imshow(image)```
*Thread Reply:* @Ben Weinstein this is great. Will follow up, thank you.
I know i've had meetings with a number of people here on annotation platforms and workflows. Just FYI i just sat through the 45 minutes on azure data labeling, its feeling like something promising https://docs.microsoft.com/en-us/azure/machine-learning/how-to-create-labeling-projects. I'll probably be giving it a try in a few months if anyone wants to come back to this thread. We currently use a combination of Zooniverse and QGIS.
*Thread Reply:* I have shown a demo of the beta of AIDE v2.0 to some of you a couple of months ago. To the others (and to you, if you don’t mind a refresher): would you be interested in a demo session? In that case I’d gladly prepare one and communicate a day and time. Please quickly respond below or mark this comment with a ✅ if you’d like to see a demo. Thanks!
*Thread Reply:* @Benjamin Kellenberger fantastic news! Keen to set up with our models asap ..
*Thread Reply:* Like training data? Or a prebuilt model? What kind of objects?
*Thread Reply:* You might check out AIDE: https://github.com/microsoft/aerial_wildlife_detection
*Thread Reply:* First train, then identify. Marmot burrows in high altitude pastures
*Thread Reply:* Thanks a ton for the resources Carly, will have a look
*Thread Reply:* Downloading the resources from GitHub as we speak. Excited 😊
*Thread Reply:* @Elizabeth Bondi has a series of work on detecting humans and animals from thermal infrared images!
Paper: https://projects.iq.harvard.edu/files/teamcore/files/2018_35_teamcore_spot_camera_ready.pdf Dataset: https://sites.google.com/view/elizabethbondi/dataset
*Thread Reply:* Thanks @Lily Xu! Please let me know if you look into this and have questions, @Koustubh Sharma.
*Thread Reply:* Hi Elizabeth, I have never used the docker, and am getting some errors when trying to run it on my computer. If not much of a problem, can I bug you sometime this week or the next for a little assistance on running the tool?
WWF’s Dave Thau is doing a series on AI for conservation and managing planetary data - https://medium.com/g-ai-a/g-ai-a-artificial-intelligence-and-planetary-scale-environmental-management-e34f2571761e
I've talked with a few people here about the bird detector for high-res airborne imagery. Here is a teaser with an early release model. It will continue to improve. All feedback welcome. https://colab.research.google.com/drive/1e9_pZM0n_v3MkZpSjVRjm55-LuCE2IYE?usp=sharing
*Thread Reply:* Antarctica, and Minnesota
*Thread Reply:* not yet. perhaps in the future. this is demo only.
*Thread Reply:* Have you tried more than those 2 resolutions? I'd be curious what the performance/input size curve looks like and whether there's any way to predict what resolution in your test data will do best based on your training data/model.
*Thread Reply:* Just illustrating what the problem is. Here is with no data augmentation on a held-out dataset at new resolution. It makes mini ducks. Prediction in blue.
*Thread Reply:* Working on even heavier zoom and cropping.
*Thread Reply:* Very interested, following along for more updates 🙂
*Thread Reply:* Some of my favorite results: • The gap between supervised and self-supervised performance on state-of-the-art fine-grained datasets like iNat21 is huge (~30% top-1). • There isn’t much benefit to pretraining on more than 500k images. • Adding more pretraining images from different domains does not necessarily lead to more general representations.
*Thread Reply:* Awesome! Seems to be a great work indeed; I’m looking forward to reading it. Thanks @Elijah Cole (Deactivated)!
*Thread Reply:* I don’t know how prevalent they are, but I know there are some images with various watermarks in standard datasets like ImageNet and Places365. People don’t seem to worry about them too much in practice.
@Sara Beery didn’t you look into this for camera trap images once?
*Thread Reply:* A bit! camera traps frequently have these bars at the top and bottom that are distinct to camera type. I've experimented with cropping them out and with leaving them in and found it didn't make much of a difference.
*Thread Reply:* Thanks a lot for your inputs. I'm assuming if the frequency, appearance and location of the marks, are more or less equally distributed across classes it will not affect global evaluation metrics. Or if their frequency is so low that it is drowned in the standard error margin. In my case, I fear, they might end up being discerning features. Will try some experiments..
*Thread Reply:* In my case, I think they are discerning features, but that there are a bunch of other discerning but correlative features the model can focus on just in the static backgrounds.
*Thread Reply:* So with or without them, the model finds it easy to overfit
*Thread Reply:* true, a lot of overfitting opportunities for the model 🙂
*Thread Reply:* Interesting, Sara. I was concerned about these on camera trap images, but have just ignored them rather than try to crop them out. In my case, different camera brands are deployed at different sites and different sites have a somewhat different suite of species, so I thought this may increase overfitting. Irrelevant camera information on the photos correlates with species occupancy/prevalence.
*Thread Reply:* yes, the bars correlate with species prevalence, but so does the entire background of the image and underlying hardware-specific image statistics? So I guess my point is that yes, models can focus on the bars but without the bars they still have a lot of opportunity to overfit - which is my hypothesis as to why removing the bars didn't do much
*Thread Reply:* Ok, I take it back, if you add a project that only has images of one species, and it's a camera type that isn't represented previously, then yep. Massive overfitting to the logo. I think probably in previous experiments I was doing there was a lot more variability in species per camera type so it was less of a problem.
*Thread Reply:* thanks @Sara Beery makes sense.. if there's a constant signal for a particular category it will be picked up by filter weights.. i'll update this space once I have some curated data (long task)
*Thread Reply:* k, sorry if that was naive idea, but its interesting how often I feel like the unlabeled auxiliary species is actually better quality than the, often rarer, labeled, target species. Like this one where, i'm not sure the target is, but I hear glossy backed thrush? and a tapaculo?
*Thread Reply:* lol, sorry, didn't mean to :https://xkcd.com/1831/ i'll go back to images.
*Thread Reply:* We’ve actually been hard at work on this problem! Stay tuned for some cool stuff rolling out over the next few weeks and months
Here’s a link:
https://arxiv.org/abs/2106.09708
Happy to discuss!
*Thread Reply:* The code is now available as well if anyone wants to play around with it:
https://github.com/elijahcole/single-positive-multi-label
Super excited to announce the launch of Sound ID for Merlin. Simply hold your phone up, press record, and Merlin will help you ID who’s singing in real time! If you happen to be in the US or Canada, download it today and try it out in your own backyard. Over the next months (and years!) we’ll be expanding the feature to cover more species and regions. https://merlinbirdid.page.link/sound-id
*Thread Reply:* @Ben Weinstein it doesn’t cover S. ecuador yet but we’ll get there soon!
*Thread Reply:* awesome. I'm going outside to try it now.
*Thread Reply:* i'm also trying to do my saw-whet call to see what happens.
*Thread Reply:* lol BENNNNN we're in a workshop 🤣
*Thread Reply:* (ie I want to go outside and try it too)
*Thread Reply:* okay @gvanhorn I have some questions. First, amazing, congrats, truly a remarkable accomplishment.
*Thread Reply:* Can I opt in to let Cornell save my recordings to improve future models?
*Thread Reply:* Could I annotate them here, like, select 'yes' or no for a given species?
*Thread Reply:* like it was perfect on lesser goldfinch, but then tossed a hutton's vireo in there (it was the same call), you can imagine users contributing in a weak supervised way.
*Thread Reply:* @gvanhorn Is this feature of Merlin related to BirdNET?
*Thread Reply:* but watching it click on a song sparrow (nice and close) is a warm and fuzzy feeling.
*Thread Reply:* Uploading recordings to the Macaulay Library will hopefully be available by this fall. I think it will also support uploading photographs, and could have tighter integration with eBird.
*Thread Reply:* brilliant. The data gathering aspect of this is so valuable, like tesla watching you drive.
*Thread Reply:* it did a pretty good job at a distant set of species, i was impressed with the western scrub jay it grabbed from several houses over.
*Thread Reply:* what was the key innovation that made this happen, $ for developers? More clean data? Algorithmic improvements?
*Thread Reply:* And then plenty of headroom for improved performance. One limitation that I’ve been trying to work around within Merlin is that taxonomic predictions isn’t well supported by the app UI. So we’ll hopefully keep improving upon egregious mistakes, but then more fine-grained mistakes will require a bigger UI update that let’s me return taxonomic probabilities.
*Thread Reply:* Sound ID benefitted from some fore-runners like BirdNet and BirdVox (@Ed Miller this sort of answers your question), so that got us up and running pretty quickly. Then it was experience with dataset building and working with experts to refine the annotations that got us to a point where we were reasonably happy with the live experience. Certainly the experience with Seek helped out here
*Thread Reply:* @Ben Weinstein I guess the answer is: right team and the right data.
*Thread Reply:* I think that's a good answer. You didn't say, we need new and interesting and complex models.
*Thread Reply:* when I give talks I always ask, do we need new models, new theory or new data.
*Thread Reply:* and clearly all three are nice.
*Thread Reply:* but where should we focus/
*Thread Reply:* I feel like Grant in particular is really fantastic at curating accurate labeled data with experts in the loop, and not just the data collection but getting these difficult labels to be accurate was key.
*Thread Reply:* A teaser blog post (about future blog posts that will have more details….) can be found here: https://www.macaulaylibrary.org/2021/06/22/behind-the-scenes-of-sound-id-in-merlin/
*Thread Reply:* @gvanhorn if there is a way for me to demo a pack for the tropics when I go, happy to give feedback. Not sure if that's feasible/useful.
*Thread Reply:* It is possible, you can join the Merlin Beta here: https://merlin.allaboutbirds.org/beta/ The Beta testers will get the upcoming packs first, but currently the model in the Beta and the Production app are the same.
*Thread Reply:* i've been playing with this a bit more, really interesting. Is there a long term interest in making an API so that scientists can query recordings against the model?
*Thread Reply:* Good idea! Yes and no. Yes: definitely interested and I can see the benefit it would provide to scientists. No: I don’t have the funds queued up to support that kind of API, although I don’t think it would cost too much run a bare bones / no fuss api. But we’ll see how things shake out this summer.
*Thread Reply:* I'll be there with @Elizabeth Bondi!!
*Thread Reply:* I've never done a poster session. Let alone a virtual session on Gatherly! 😧
*Thread Reply:* We aren't totally sure what to expect either, but it should be fun 🙂
*Thread Reply:* I will also attend, no idea how this virtual poster session could go, let’s see!
Came across this project: https://seabee.no/ Using drones and AI to monitor and map the biodiversity of the Norwegian coastal zone, specifically used for bird counting. Super cool!
*Thread Reply:* any idea who is behind this? No contact on the website? did i miss it?
*Thread Reply:* https://www.linkedin.com/in/jake-scubotics/
*Thread Reply:* funny, that's my institution, never heard of it.
Can our datasets be of value to you? A few project updates
The University of Hong Kong has just developed an app called ‘Saving Face’ for i-id of napoleon wrasse:
https://onlinelibrary.wiley.com/doi/abs/10.1002/aqc.3199 https://www.scmp.com/magazines/post-magazine/short-reads/article/3134123/facial-recognition-app-used-protect-endangered
The team has developed other ML applications in conservation such as: https://www.clearbot.org/ https://openoceancam.com/
*Thread Reply:* Yep. If you've got the PSD file, exporting to SVG and then parsing shouldn't be too hard.
*Thread Reply:* thanks. i'll update here. Right now i've got the JPG, but requesting the original.
https://gitlab.kitware.com/viame/bioharn/-/blob/master/dev/data_tools/read_ps_count_jpeg.py
Hey Jon 🙂, I think storing annotation data inside the image itself it probably not a great idea. But I do think existing annotation formats are lacking. I do like the general idea behind MS-COCO because it provides a relatively intuitive way to specify annotations, and I've been working on an extension of it called KW-COCO, which adds support for images-sequences, multispectral imagery, and richer annotations (like polygons with holes):
https://gitlab.kitware.com/computer-vision/kwcoco
The kwcoco CLI has a command that is supposed to help grab data and convert it into kwcoco. Currently it support for cifar, domainnet, sapcenet7, and camvid, but I'm looking into adding datasets from lila.science. I also recently found out about a library that scikit-image uses for fetching data files called "pooch": https://github.com/fatiando/pooch . Ultimately, it would be really cool to have an IPFS solution for all of these public / benchmark / challenge datasets.
Notion’s ‘Sessions’ will be available free to all Beta users who join and complete below checklist once Sessions launches July 12 2021 🙂
*Thread Reply:* Welcome!! One way you can do this, and I think what a lot of folks here have done, is find a PhD advisor who will support this kind of interdisciplinary work within a traditional EE/CS type program
*Thread Reply:* For example, @Elijah Cole (Deactivated), @gvanhorn and I all did/are doing our PhD's with @Pietro Perona (who is the best, thanks Pietro!!) who is supportive of our work on different aspects of real-world problems in the natural world and other social good domains like medicine. Milind Tambe advises @Lily Xu and @Elizabeth Bondi and has a similar passion for work that has positive societal impact. If you're more on the hardware side one person you could reach out to to get a better sense of the lay of the land might be Andrew Schultz, who is a PhD student in MechE at Georgia Tech and does awesome conservation-focused research and education (https://streamerlinks.com/StreamingScience)
*Thread Reply:* Hi Eric! I am just starting the transition from working as a software engineer to becoming a PhD student (with Pietro 🎉 !). The process of finding programs and debating PhD vs. private industry work is very fresh in my mind. While not entirely up to date, this site has a number of researchers in this intersection. Alternatively, in the private industry, the Work On Climate slack channel is a good place to lurk for companies trying to solve climate issues (not conservation-focused, but there could be some overlap in interests).
*Thread Reply:* Thank you so much @Sara Beery and @Suzanne Stathatos! All your resources are fantastic, and CompSustNet in particular looks promising!
*Thread Reply:* Welcome @Eric Greenlee - I believe @Ben Seleb is doing a graduate project in conservation hardware. Worth giving him a shout.
Also, Shah Selbe is into field IOT hardware. Maybe his work can give you some leads
https://medium.com/conservify/bringing-conservation-technology-to-life-eadd93bfa3af
*Thread Reply:* Thanks @Mike C! I reached out to Ben and I actually to Shah 6 months ago about FieldKit
*Thread Reply:* I don't know of a public one, but I was JUST talking to someone who is interested in this on Friday for Ag applications (detecting invasives)
*Thread Reply:* yeah! That's pretty much the exact same idea. If something comes through the grapevine I'd love to hear about it
*Thread Reply:* @Chandra Krintz and @rich wolski this is similar to what we were talking about Friday, right? Do you guys know of any public datasets?
*Thread Reply:* let me know too, I have collaborators in switzerland working on microscope images.
*Thread Reply:* I don't anything annotated and ready to share yet, but I'll update (probably be like a year)
*Thread Reply:* you should email https://www.luerig.net/
*Thread Reply:* I'll do that! Thanks 🙂 The dataset doesn't actually have to be labeled
*Thread Reply:* @Sara Beery @Stefan Schneider @Ben Weinstein Hi All. Sadly, while we know of a research team that is working on insect identification I don't think they have made an image data set public. We will check to see if they have an interest in doing so.
I know of this honeybee tracking dataset: https://zenodo.org/record/4400651#.YO2kE3VKi54 which corresponds to the paper/code described on this github page: https://github.com/vladan-stojnic/Detection-of-Small-Flying-Objects-in-UAV-Videos
*Thread Reply:* thanks for pointing this one out!
Hey everyone! Happy to be a part of this community. I wanted to share a project we are doing about conserving monarch butterflies. These tiny creatures travel over 3000 miles every year to their overwintering sites in Mexico (and some head to California). As monarchs stay in extremely dense clusters, it is very difficult to count them. Currently, people simply estimate size of the population every year by looking at the total hectares occupied. Unfortunately, this can be extremely error-prone. We collected images of monarchs and showed how simple crowd counting techniques using deep nets can be used to accurately estimate counts of butterflies in dense clusters. Read the paper here: https://www.biorxiv.org/content/10.1101/2021.07.23.453502v1 Our dataset is open for anyone to use so if you are interested, please reach out. Also, if you know anyone in CA or Mexico who might be interested in trying this out in the field, please let us know and we will be happy to help!
*Thread Reply:* Thank you!!! This was a joint effort with @Elijah Cole (Deactivated) and was wonderfully advised by @Kevin Winner, I learned a TON while writing it 🙂
@Siyu Yang I enjoyed your recent presentation and Q&A on the MegaDetector! Very useful. https://youtu.be/LUkQVARAVFI
*Thread Reply:* Thanks! Glad people find it helpful
*Thread Reply:* Only thermal from this project (if useful)
*Thread Reply:* http://lila.science/datasets/arcticseals and http://lila.science/datasets/noaa-arctic-seals-2019/. I'm not certain if both of these have polar bears, the first one should but unfortunately not many and you would need to download a lot more data to access them. Send me a message and I can package up some polar bear data seperately. Also these are aerial images so no close ups of polar bears. Would love to hear what you are working on if you want to share! 🙂
*Thread Reply:* @Yuval Boss Ah, sounds great but unfortunately aerial photos won't do. I am working on face detection and recognition so need some close up shots! Similar work to the BearID project. I did find this dataset which contained ~900 images of polar bears so that's a good starting point 🙂
*Thread Reply:* Awesome! If you come across any aerial polar bear data don't hesitate to share 🙂 we only have a few hundred examples and could really benefit from more
*Thread Reply:* Thank you, I will get in touch!
*Thread Reply:* Awesome! Welcome to absolutely beautiful old Dunedin! 😄
*Thread Reply:* hey @HM van Zyl - also from CT here! Are you interested in collaborations with academics? I've recently joined stellenbosch uni, let me know if you'd be keen and there should be a new cohort of MSc students in engineering looking for projects soon
*Thread Reply:* Hey Emmanuel! I'd love to do a quick meet. I'll pm you my email and number then we can get started! Perhaps someone in the channel could give us an idea of something cheap and dirty we could attempt. I think the Stellenbosch campus is ripe for some bird monitoring! I'll take this over to random.
*Thread Reply:* I have tons of ideas, literally a whole list of potential MSc topics! 🙂 So yes, I think collaborating with the zoological department at SU and the data science school which I work in would be a great 3-way collaboration. I have a bunch of audiomoths and field recorders. So yes, please do PM me your details
Hi; I too joined after the excellent interactive presentation by @Sara Beery yesterday. I work with DynAIkon which focuses on AI solutions for the camera trap world. We are part of a large EU consortium which covers different facets of conservation (focused on observations of plants and animals) and citizen science : https://cos4cloud-eosc.eu/
Hello! Our group (Whale Seeker) detects marine mammals from imagery using AI. We are trying to create educative and thoughtful content about AI so non-experts can better understand the how and why. We frequently come across the attitude from possible clients that “AI is too complicated and expensive so we’ll substitute crowdsourcing/citizen science instead.” We wrote a blog on this topic that we’d love your thoughts on if you have time. https://www.whaleseeker.com/blog/widening-the-bottleneck-can-citizen-science-accelerate-conservation. Thank you in advance!
*Thread Reply:* Love your blog posts. I would love to share a summary version linking to the full article (backlinked and attributed to you) on the Open Ocean Camera Medium publication - can we get permission to? 🙂 @Emily Charry Tissier
Stumbled upon the ARM AI tools catalogue while browsing ConservationX. Might be a resource for those looking for tools, and an advertising channel for those looking to spread word about their solutions?
https://www.arm.com/why-arm/partner-ecosystem/ai-ecosystem-catalog/conservation-x-labs
*Thread Reply:* I work for Arm. Feel free to message me if you have any questions!
*Thread Reply:* There are a lot of accessible datasets/challenges on kaggle via the FGVC workshop competitions! https://sites.google.com/view/fgvc8/competitions
*Thread Reply:* Hey Carly this sounds brilliant! The link does not work for me for some reason though. Wondering if the link is deprecated or whether there is a new one at all? Many thanks
*Thread Reply:* Hi @Monty Ammar! The directory now lives at https://conservationtech.directory
Hi all! We at DrivenData just launch a new machine learning competition for wildlife depth estimation in camera trap videos. It features a great dataset of labeled #camera_traps videos from MPI-EVA and Wild Chimpanzee Foundation researchers.
We'd love for you to participate and help spread the word with anyone you think might be interested! Thanks!!
*Thread Reply:* @Greg Lipstein how was the ground truth collected?
*Thread Reply:* Oh nevermind, I found it 🙂 https://www.drivendata.org/competitions/82/competition-wildlife-video-depth-estimation/page/392/#about-the-data
*Thread Reply:* Thanks @Sara Beery! Check out About the Data section here https://www.drivendata.org/competitions/82/competition-wildlife-video-depth-estimation/page/392/
*Thread Reply:* So, along those lines, any sense of what the accuracy in this manual labeling is?
*Thread Reply:* @Elijah Cole (Deactivated) and I captured a similar thing for some of our cameras, but I got the sense that there was a lot of room for human error/subjectivity to creep in.
*Thread Reply:* That would be a good question for the MPI-EVA team. From what we’ve heard, that's right; it's often a time-consuming, manual, and error-prone process to attach these distance labels. That said, some noise in labels is to be expected, and the difficulty labeling provides some of the value from having ML solutions that can help. The scale of the data also helps here, especially to the extent the noise is random
*Thread Reply:* Absolutely! I'm curious if there's an upper bound on performance based on label error, and/or if the error is biased for a given labeler (ie maybe I'm likely to always label closer than reality, for example)
*Thread Reply:* Another interesting thing to look at would be agreement between labelers, did multiple experts label each image?
*Thread Reply:* (I realize these are not necessarily things you have answers to, just thinking out loud 🙂 )
*Thread Reply:* > I realize these are not necessarily things you have answers to, just thinking out loud 🙂 Totally! Those are all good thoughts
*Thread Reply:* @Greg Lipstein I see that you tried our monodepth2 code in the example image - very cool. Have you tried Midas, it is trained on much more diverse data? https://github.com/isl-org/MiDaS
*Thread Reply:* We did! I haven't seen Midas but I'm also not the right one to ask 🙂 . Much better to check with our (fantastic) data scientist @Emily Dorne
*Thread Reply:* @Oisin Mac Aodha I have seen the paper but haven't yet tried out the code! Very cool to see transformers being used in this space
*Thread Reply:* Hi @Greg Lipstein! What does the ground truth distance mean when multiple animals are present in the image? I couldn't find any reference in the resources listed on DrivenData.
*Thread Reply:* @Ștefan Istrate Wherever possible, frames with multiple animals present have been excluded from the labels that participants have to predict. However, the hand-labeled ground truth is not 100% comprehensive and there are some unlabeled animals. We used the megadetector model to filter out any frames with multiple high-probability bounding boxes, but the predictions are not perfect and a small number may have slipped through - looks like you found a good example of this. The estimated bounding box given is the one with the highest probability, but may not correspond exactly to the label. These cases should be very rare in the data!
IJCV Special Issue focusing on Animal Tracking and Modeling.
*Thread Reply:* My group is working on this problem a bit, would be happy to share if interested. There are a few interesting papers in this area as well:
Semantic Redundancies in Image-Classification Datasets: The 10% You Don’t Need https://arxiv.org/pdf/1901.11409.pdf A critical look at the current train/test split in machine learning https://arxiv.org/pdf/2106.04525.pdf Learning From Less Data: A Unified Data Subset Selection and Active Learning Framework for Computer Vision https://arxiv.org/pdf/1901.01151.pdf Do we train on test data? Purging CIFAR of near-duplicates https://arxiv.org/pdf/1902.00423.pdf Apple Neural Hash (CASM) Report https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf
Also a few commercial options:
*Thread Reply:* Thanks! Lots of good stuff in there. I thought about using Apple's Neural Hash for this... I wonder how effective it would be. Do share your work! I am interested to learn more about this.
*Thread Reply:* I have used the perceptual hash for this, with decent though not 100% perfect results: https://github.com/knjcode/imgdupes
*Thread Reply:* That looks great, thanks! Will give it a go.
*Thread Reply:* When you say "share" images between train/test, what exactly do you mean? It's a really interesting question because there is a big difference between finding exact duplicates, near duplicates (smooth augmentations such as color shifts, translations, crops, rotations...), and finding similar images which are not reachable at all in augmentation space all but are so semantically similar that you would never want to both train/test on them.
*Thread Reply:* I was initially only considering exact duplicates but now you've opened my eyes! I'd love to get a measure of near duplicates and similiar images but I wouldn't consider it a priority for what I am doing just yet.
*Thread Reply:* My first instinct would be to hash them. I've never used it, but this might be worth checking out: https://github.com/JohannesBuchner/imagehash
*Thread Reply:* If you're using image hashes, specifically apple's neural hash, be aware of this https://blog.roboflow.com/neuralhash-collision/
Focus issue from AIHub on "Life on Land": https://aihub.org/2021/09/10/focus-on-life-on-land-call-for-contributions/
*Thread Reply:* this is a wonderful initiative, Sara! so glad that you're doing this!
*Thread Reply:* This is so awesome, Sara! Will definitely spread the word!
@Gracie Ermi and I just migrated our Conservation Tech Directory site to a new platform that is more user-friendly and aesthetic. It has a snazzy link title now too! - conservationtech.directory. Check it out!
This update also comes with ~30 new entries as well, bringing the total count to 447 now. As before, use this Google form to submit resources that should be on here but aren’t yet. And downloadable PDF of the directory can be found on FigShare.
*Thread Reply:* Also, please feel free to use the google form to submit corrections to resources that are already in the list. We want this to be as accurate as possible, so we’re all ears if you have a correction or a piece of info to fill in!
*Thread Reply:* I do not know of any ML ready datasets for these metrics.
*Thread Reply:* I could recommend about how one could be created, but I would be surprised if they existed.
*Thread Reply:* there was a student from geohackweek I taught at UW who was using images from industrial traffic from west africa to try to estimate carbon intensity and emissions. I could find her name.
*Thread Reply:* Hi Chris! Agreed with Ben - there doesn't exist an ML ready carbon dataset ,but some people are working on it. This community has assembled a GitHub list on processed and raw datasets in forests which be helpful for you: https://github.com/blutjens/awesome-forests
If you're just looking for nice climate-tangent ML ready datasets ,the Kaggle datasets in this list would be the perfect fit for you. Lmk if that helps
*Thread Reply:* Which scale are we talking about? Handheld cameras or satellites?
*Thread Reply:* @Ben Weinstein: Thanks for confirming my hunch that such datasets are hard to come by.
@Björn Lütjens: Thanks for that GitHub link! A couple of the deforestation tracking ones look relevant.
@Daniel Davila: either!
*Thread Reply:* For emissions in particular, I can think of a few leads... all commercial companies though. There are a few groups working on co2 and methane plume detection. The two leaders I am familiar with (having worked for the former) are SwRI with SLED and FLIR systems with their OGI line of cameras. They both have extensive methane plume datasets, but whether they would allow you access to them is a different story. There are some air/space-borne flavors of this tech offered by other (mostly proprietary) sources like GHGSat and the EDF's upcoming MethaneSAT mission, which we'll have to see if they make that data public and annotated.
*Thread Reply:* @Daniel Davila: Yeah, I'm aware that many commercial companies working on this space. Just to confirm - you aren't aware of any publicly + freely accessible ML datasets for emissions monitoring and/or plume detection, right?
*Thread Reply:* No, sorry! Somebody should fund the creation of one
*Thread Reply:* Global Forest Watch tracks deforestation through ML analyses of geospatial data. I’m not sure if the actual training data is public but you might try reaching out the them? If anything the folks at WRI (makers of GFW) might have leads on the types of datasets you’re looking for.
*Thread Reply:* @John Brandt is connected to this project.
*Thread Reply:* Hi Chris,
ESA’s AI4EO initiative organised its first challenge to improve emissions monitoring. It was a super-resolution task. The challenge is already closed, but maybe you can find the dataset somewhere or contact the organisers. https://ai4eo.eu/ai4eo-challenge1 https://platform.ai4eo.eu/air-quality-and-health/data
AIREO is another ESA initiative with the aim of providing “AI-ready” datasets. I think the initiative is still in its early stages. Therefore, the prototype datasets currently available are rather small and not really suitable for training deep models. https://www.aireo.net/
cheers
*Thread Reply:* just clarifying here esa -> is european space agency, and not the ecological society of america. confused me.
*Thread Reply:* Thanks @Carly Batist - I am aware of Global Forest Watch. However, yeah, I'm concerned that the publicly accessible maps are not the original "ground truth." I may consider reaching out to WRI.
@Nico Lang - Thanks for sharing the AI4EO and AIREO datasets. Will take a look.
Relevant to the above thread: https://www.frontiersin.org/research-topics/26080/forest-carbon-monitoring-and-artificial-intelligence
*Thread Reply:* Do you mean re-identify specific animals across images seen at different camera traps in a grid?
*Thread Reply:* Yes. I apologise for the vague question. I am looking for a method of identifying indivduals of a particular species, that have been previously detected(by megadetector), to map their movements.
*Thread Reply:* What species? There are methods that work well for certain types of patterned species (see https://www.wildme.org/#/ and https://sites.google.com/corp/view/wacv2020animalreid/home)
*Thread Reply:* @Jason Parham, @Maxime Vidal and @Stefan Schneider are the experts :) The first challenge is curating ground truth in order to robustly evaluate one of these methods on your specific data/population
*Thread Reply:* The family of Suidae is of interest, with Warthog being the most important. They do lack patterns, but their facial features(eg. tusk length) can sometimes be used to identify individuals.
*Thread Reply:* Cool. I'm working with elephants currently, and we're exploring tusk length and shape as a contributing identifier but it's still work in progress. Is this something humans are able to do reasonably well for warthogs?
*Thread Reply:* I am no expert in identifying Warthog, but the size of the tusk and overall body size can give a resonable indication of age, which can help in identifying individuals.
*Thread Reply:* 1. Would it be feasible to train from a checkpoint(efficientnetb0-7). If it is feasable, how many images would be required, per individual, to achieve reasonable results.
*Thread Reply:* Hey Stuart. Here's a paper I wrote that gives a 1000ft view of the field. There may be one or two techniques that standout to you from here.
https://besjournals.onlinelibrary.wiley.com/doi/epdf/10.1111/2041-210X.13133
If you're interested in state-of-the-art, you have a question of if the population is closed vs open. If you have a closed population where you can get images of every individual, you can use a traditional classifier (Conv Neural Network kind of thing). If the population is open, you'll need something like a similarity comparison network.
I have a version of this paper catered towards ecologists coming this Winter sometime. So if this one is a little too abstract, stay tuned for that.
*Thread Reply:* Awesome, thank you for your help @Sara Beery @Stefan Schneider !
*Thread Reply:* @Stefan Schneider Hi Stefan, in your paper it states that ideally, the dataset for training a re-ID system(similarity comparison), >500 individuals are needed. Would a triplet loss network still be advantageous over a CNN, in a smaller population(<100) therefore containing a higher number of images per individual.
*Thread Reply:* Just so we're on the same terms, triplet-loss describes the loss used by a similarity comparison network, which is a form of Metric Learning. You could train a similarity comparison network using a number of different losses, triplet and contrastive are the two I use in the paper.
CNN and similarity comparison networks need approximately the same number of images. The advantage of similarity comparison networks is their accuracy will transfer to previously unseen individuals, and you only need ~500 images of a few individuals, whereas a CNN ideally has 500 images for every individual and will erroneously classify previously unseen individuals as one with data
500 is a rough ballpark number. You could get away with less if you go heavy on augmentations.
If animal re-ID is what you require, best bet would be to get started. Try training a similarity comparison network using a contrastive loss (it's easier) and just see what accuracies you get. You can go from there to determine if you need additional data, more augmentation, use of the triplet-loss, etc. If you've never trained a machine learning model before, start with a basic CNN and build from there. You'll need the CNN anyways for the similarity comparison network
*Thread Reply:* Thank you very much, that is very informative.
The Alan Turing Institute are collaborating with CEFAS to run a two-phase Data Study Group (DSG) to explore and build toolkits for rapid identification of plankton using machine learning
Please feel free to pass on to your networks 😊
https://www.turing.ac.uk/events/data-study-group-november-2021
NSF call on Biodiversity on a Changing Planet: https://beta.nsf.gov/funding/opportunities/biodiversity-changing-planet-bocp
*Thread Reply:* This call also opened recently (preproposals due on Nov 5th): https://www.biodiversa.org/1772
Hi all! I’m a Lecturer in Ecological Data Science at the University of Exeter in the UK. I’m interested in lots of different things at the AI/ecology/conservation interface, but especially remote sensing of forests, ecological networks and camera traps. I’m putting together a small grant application for a 3 month project on supplementing the training datasets of species identification algorithms for camera traps with data augmentation methods, synthetic imagery and transfer learning. I’m aware of this (https://arxiv.org/abs/1904.05916) awesome paper by @Sara Beery. But are there any other examples? Don’t want to duplicate effort
*Thread Reply:* I did a follow up with a student at TU Delft where we used image-to-image translation to improve on the results with the generic synthetic data: https://arxiv.org/abs/2106.12212
*Thread Reply:* talks with @Sara Beery inspired a tree crown paper that had a similar vibe. using LiDAR to create weak labels for RGB learning. https://www.mdpi.com/2072-4292/11/11/1309
Hello everyone! I am a sixth-year PhD student from UC Berkeley. We have just published a paper on Nature Machine Intelligence about deployable wildlife AI recognition systems with efficient humans in the loop and imperfect models! Please check it out if you are interested! This project is moving towards practical deployment in Africa. We hope this work can become an important step to the AI solutions for real-world conservation problems with imperfect AI models. I am very happy to answer any questions you have! Here is the link to the journal webpage (with a preprint link incorporated in case you don't have subscriptions): https://www.nature.com/articles/s42256-021-00393-0 Thank you very much!
*Thread Reply:* Welcome, Matt! Are you by chance working with Prof. Rebecca Hutchinson? If not, you may be interested in her work at the intersection of ML + ecology 🙂
*Thread Reply:* Hello from Corvallis! ❤️ OSU
*Thread Reply:* I am not working with Rebecca. However, I took a few courses with her during my Master's. She is a great teacher and really knowledgeable about the intersection of ML and ecology. Especially about ML methods applied to distributions and interactions.
*Thread Reply:* Welcome, Matt! You might also be interested in Justin Kitzes from UPitt who is doing very similar projects on bird calls and multi-label classifications.
*Thread Reply:* ^ And Tessa Rhinehart, who is a PhD student in that lab! Also might be worth talking to Stefan Kahl, who runs BirdNet (from Cornell Lab of O)
*Thread Reply:* And @gvanhorn who just released sound ID in the merlin app!
*Thread Reply:* Thanks for the recommendations everyone. I'll reach out to a number of these researchers. Digging through some of the recent literature I get the feeling it might be time for a small symposium to gather people working through similar problems.
*Thread Reply:* I've found it's hard to get datasets with humans. Because these are usually stripped.
*Thread Reply:* The humans that are in there are not respresentative, because it's usually people deploying or maintaining the cameras, very close up
*Thread Reply:* In what context do you want to use the model?
*Thread Reply:* I know the megadetector is pretty good at picking out humans.
*Thread Reply:* The human data is usually stripped before publication for ethical reasons, because the humans in the images have not given consent (and frequently are not aware their photo is being taken). MegaDetector already does a pretty good job of what you're describing, try it out here: https://github.com/microsoft/CameraTraps/blob/master/detection/megadetector_colab.ipynb
<a href="https://github.com/microsoft/CameraTraps">microsoft/CameraTraps</a>
*Thread Reply:* Thanks both. I want to train a small SSD model for edge deployment on a video camera trap. We are interested in human rejection from our captured images. Perhaps megadetector could be used to generate my training data... 🤔
*Thread Reply:* Yeah I use it for weak labeling all the time!
*Thread Reply:* @Elizabeth Bondi has a BIRDSAI dataset with exactly this, for thermal imagery! https://sites.google.com/view/elizabethbondi/dataset
*Thread Reply:* Thanks @Lily Xu! Please let me know if you have any questions, @Ross Gardiner
*Thread Reply:* Check out WILDLABS, an online conservation tech community! And the Conservation Tech Directory for examples of resources/companies/organizations working in geospatial applications for the environment. Search ‘geospatial’ or ‘remote sensing’
*Thread Reply:* Thank you, @Carly Batist ! ☺️
The Conservation Tech Directory has gone past the 500 mark!🎉🎊 Now at 512, to be exact. Thanks to all who have contributed new entries & corrections on our Add/Update a Resource form (keep them coming 🙂). AND my co-developer @Gracie Ermi has given us a logo (see below)!
Hi folks 👋 I'm a Data4Good Lead with a Data science background at Appsilon. I run there an internal program where we support sustainability initiatives. One of my colleagues @Jędrzej Świeżewski is already here for some time. We are the team behind Mbaza project, AI tool that helps classify animals from camera traps deployed in National Parks in Gabon. Next year we will have a strong focus on biodiversity. I'm here to learn about challenges that we could help solving with our team skills (ML Vision and data viz) and network with people from biodiversity space. I'd especially would love to meet researchers who's research would benefit from cooperation with us. If you want to grab a virtual coffee with me, send ma a DM. I'd love to chat!
*Thread Reply:* ようこそ! Great to see you here! 😄 And a great initiative too; I am likewise interested in getting to know more about the potential applications of vision(ary) work for conservation & Co.
*Thread Reply:* @Rowan Converse What's the database size? For scientific research purposes, a DOI-granting repository like Zenodo.org (which is built on CERN's Data Center) is probably preferable.
*Thread Reply:* I believe up to 50 GB per 'dataset' is free, and there is not a limit on how many free datasets (https://help.zenodo.org/). They encourage you to contact them for larger use cases. You've probably already thought through copyright for distribution and possible concerns about geolocation of rare or protected species.
*Thread Reply:* There's also LILA.science @Dan Morris
*Thread Reply:* As Sara suggests, we host some data like this on lila.science, e.g.:
https://lila.science/datasets/aerial-seabirds-west-africa/
If you're interested, drop us an email at info@lila.science. Thanks!
*Thread Reply:* LILA looks really cool, I hadn't seen that. No doubt there are some significant performance advantages in using Azure-based hosting, especially if your data is in the TB+ range (like those NOAA seals). It's generous of Microsoft to host such a resource. All the same, I do think it would be nice to see modern metadata standards, such as the FAIR principles, for data intended for scientific archives.
*Thread Reply:* For instance, for that seal image data, NOAA maintains a rich metadata record, https://www.fisheries.noaa.gov/inport/item/63322, which includes schema.org markup, allowing it to be included in linked-data indexes like https://datasetsearch.research.google.com or data.gov. DOI-granting repositories also integrate into publication services for purposes of citation metrics to the original data, track versions, and must meet a high standard of archival persistence for scientific research. These may be considerations if citability and discoverability are important, particularly if LILA grows to index a more datasets than can be easily browsed manually.
*Thread Reply:* @Rowan Converse as part of the bird paper, we placed your portion on zenodo. See newmexico.zip https://zenodo.org/record/5033174#.YYBggNbMKlo I recommend zenodo for the rest.
*Thread Reply:* Thanks everyone for the replies, this is great!
*Thread Reply:* Also, figshare waives dataset size limits if it’s set as public (good for large datasets)
*Thread Reply:* Does anyone have experience/recommendations for archiving larger data sets — on the order of, say, 2-3 TB?
I checked with zenodo (a while back) and there was no mechanism that would work for both my institution and theirs to provide payment for the extra resources needed for a data set of this size, and I understand that figshare has a limit of 1TB.
*Thread Reply:* There are several datasets on LILA that are many TB. I would reach out to Dan Morris (email info@lila.science).
*Thread Reply:* I can ask Dan this, but do you know if LILA grants DOIs?
*Thread Reply:* I'm not sure. If not I wonder if that's something that could be set up.
*Thread Reply:* Oh really, I didn’t know figshare had a 1TB limit? (Though I’ve not actually tried) I know they have a ‘figshare+’ that allows up to 5TB but that requires a one-time data publishing charge (so you may run into the same problems you had with zenodo)
*Thread Reply:* Thanks! I’ll look into the details of figshare. My information came from a publisher’s page and may not be up-to-date. A charge could be OK, the issue with zenodo was more that they didn’t have an easy payment mechanism.
*Thread Reply:* LILA doesn't generate DOIs, plus if I told you we did, I would then immediately tell you not to use any DOI that I generated for you. 🙂 IMO you want to be in total control of any DOIs on which you take dependencies (e.g. by embedding in an archival publication).
[Now I will drift from facts about LILA into philosophy about DOIs...]
Regardless of where you host data, unless you are 100% sure your host will be there in 10 years, my personal recommendation is that you administer your own DOI.
*Thread Reply:* @Jorrit van Gils maybe we should have a quick chat. Sound like we're working on similar things (in the Netherlands)!
Hello everyone! I recently joined AI for Conservation and am excited to be part of the community. I am a senior in college interested in pursuing a PhD in AI for Conservation. As I haven’t found any compiled list of academic labs pursuing this kind of work, I thought that we could create one for the community. Please add any academic labs you know of working on AI for Conservation to this spreadsheet!
*Thread Reply:* https://www.compsust.net/
*Thread Reply:* I’m not sure who to add but you should be aware of this work: https://www.wur.nl/en/show/Imagine-all-the-animals-living-life-in-sociable-groups.htm in the Netherlands
*Thread Reply:* Tagging some of our awesome team members! @Peter Bull @Emily Dorne @Greg Lipstein
*Thread Reply:* has this been used for underwater ecosystems?
*Thread Reply:* Not yet—do you have more info on how the data is collected? I expect that since some of the parameters are tuned 1 minute long videos from terrestrial motion-triggered camera traps there is likely a fair amount of work to tune the models to the underwater setting.
*Thread Reply:* Hey Hemal.
We often have lots of dataset papers at the FGVC workshop at CVPR. e.g. from last year https://sites.google.com/view/fgvc8
*Thread Reply:* The Biodiversity Data Journal could be an option: https://bdj.pensoft.net/journals.php?journal_name=bdj
*Thread Reply:* Hi @Hemal Naik, Nature Scientific Data (https://www.nature.com/sdata/) is a great place and so is the NeurIPS database track. We just published one at NeurIPS so I would be happy to answer any questions that you might have.
*Thread Reply:* Hi @Ayan Mukhopadhyay thanks for the offer. Would definitely like to get in touch. Can you share link of your paper. I assume the paper would also have communication email. Looking forward.
*Thread Reply:* Sure, here you go https://ayanmukhopadhyay.github.io/files/neurips2021.pdf
Clare Barclay, Chief Executive Officer at Microsoft UK, said: “The untapped potential of AI and machine learning can help solve some of the world’s most complex environmental challenges. Our first-of-its-kind multispecies AI model Project SEEKER can help tackle the wildlife trafficking trade, while protecting animal ecosystems. The importance of collaboration and partnership with more organisations couldn’t be greater as we look to protect the environment and the world’s most endangered species.” https://news.microsoft.com/en-gb/2021/11/18/first-of-its-kind-multispecies-ai-model-to[…]egal-wildlife-trafficking-is-ready-to-roll-out-to-airports/
Thanks to @Lucia Gordon’s great idea to compile a list of academic labs doing conservation tech research, @Carly Batist and I have added a new “academic lab” tag to https://conservationtech.directory and a number of new entries in that category (we’re up to 568 total entries now 🎉). Know of any university groups working on conservation tech that aren’t included? Let us know so that we can get them added! And please keep letting us know about any corrections that need to be made to anything in the directory. Thanks everyone!
*Thread Reply:* Thank you Lucia for the idea and Gracie and Carly for the implementation!! What a wonderful resource (I wish I had this 4 years ago when I was applying to grad schools!)
Climate Change AI is excited to announce a call for applications for their upcoming summer school on Climate Change and Artificial Intelligence in 2022. This summer school (to be held virtually on weekdays between Aug 15th - 26th 2022 (time zone TBD), is designed to educate and prepare participants with a background in artificial intelligence and/or a background in a climate-change related field to tackle major climate problems using AI.
*Thread Reply:* Climate Change AI Summer School 2022 Virtual: weekdays between Aug 15th – 26th, 2022 (time zone TBD) Website: https://www.climatechange.ai/events/summer_school2022.html Application deadline: December 17th 2021 Contact: summerschool@climatechange.ai The Climate Change AI summer school is designed to educate and prepare participants with a background in artificial intelligence and/or a background in a climate-change related field to tackle major climate problems using AI. The summer school aims to bring together a multidisciplinary group of participants and facilitate project-based team work to strengthen collaborations between different fields and foster networking in this space. The first part of the summer school will consist of a mix of lectures and hands-on tutorials organized into two tracks, one focused on AI fundamentals and one focused on climate change. In both tracks, the program will provide an overview of machine learning applications in a broad range of climate change-related areas. This includes covering foundational machine learning methods and state-of-the-art tools, while underlining their advantages and limitations, and describing how they can be used in practice to address the climate crisis. The second part of the summer school will consist of a collaborative project at the intersection of climate change and machine learning. Participants will work together in multidisciplinary groups under the guidance of a mentor to develop AI-based solutions for climate change problems. The cohort will be composed of applicants from complementary areas of study/work, to be selected on the basis of their background and experience as well as their motivation for joining the summer school. Applications are due by: Dec 17, 2021 23:59 AOE (Anywhere on Earth, UTC-12). To apply, please submit your application through this form: https://forms.gle/9eSETySjVvdzkfiM6 Admission notifications will be sent out during the week of Feb 21, 2022. The summer school is free to attend. Applicants who are accepted will be asked to confirm their attendance for the entire duration of the summer school. This course will be instructed by members of CCAI and world-renowned experts in ML and Climate Change.
For further information please check out the summer school website or contact summerschool@climatechange.ai
@Dan Morris hey Dan, re this issue https://github.com/microsoft/CameraTraps/issues/260
I am working on developing a smart camera as part of the arribada.org initiative. The idea of the camera is to reduce human-animal conflict by providing an early alarming system for camps, villages etc.
Basically run a detection model on low cost/ low energy devices - rpi, arduino etc. and when it detects a lion or an elephant for example to send a signal to people in charge of the camp security.
We are working together with edgeimpulse.com on this project and for the idea was to avoid some manual tagging when building the ML model for the edge device.
<a href="https://github.com/microsoft/CameraTraps">microsoft/CameraTraps</a>
*Thread Reply:* Hey Benjamin, I actually came across this post today and was interested in applying. I’m still doing my masters which ends in September though, is it still alright to apply with the possibility of starting late if accepted?
*Thread Reply:* Hello Monty! At this point we have a selection of applicants to be interviewed. Also, the chances of us postponing the project to the second half of the year are unfortunately slim, as we are bound by the grantees. I’d recommend to keep an eye out for announcements, though!
Hi all! I’ve been lurking for a (long) while. I’m an associate professor in computer science at university of Massachusetts, Amherst specializing in computer vision, graphics, and ml (https://people.cs.umass.edu/~smaji/). I’ve organized the fine-grained visual recognition workshops (https://sites.google.com/view/fgvc8) and lately been looking into analyzing bird migration via RADAR data (e.g., https://www.nature.com/articles/s41558-019-0648-9). On the cv+ml side we have been interested in semi- / self-supervised learning for fine-grained classification and have organized a couple of kaggle challenges at the last two FGVC workshops.
We have been thinking about doing the same for part segmentation. Question to the community: Are there any fine-grained segmentation (part labels) datasets of animals out there? We wanted to explore few-shot segmentation tasks, but couldn’t find any datasets of birds or animals with a large number of part labels. Most of these contain keypoints or a just few parts (3-4). For comparison, there are datasets with dozens of labeled parts for human faces or cars.
*Thread Reply:* It may be a stretch, but perhaps the recent pose datasets could be of interest? It is not segmentation-ready, but it may be useful to bootstrap some segmentation ground truth.
*Thread Reply:* Hey @Subhransu Maji! Sorry for the delay here. The CUB dataset does have quite extensive part/attribute annotations for 200 species of North American bird species: http://www.vision.caltech.edu/visipedia/CUB-200.html
*Thread Reply:* Hey Sara, will this Summer school be running on a yearly basis?
Exciting volunteer opportunity on Robinson Crusoe Island, Chile.
We are pleased to announce the “Work for Humankind” partnership between Lenovo, Island Conservation and the Robinson Crusoe Island community that calls on volunteers from around the world to take part in a once-in-a-lifetime opportunity: to experience first-hand how to make a long-lasting difference with a remote island community, while working from one of the most remote offices in the world enabled by tech.
We are looking for volunteers with a range of skills, backgrounds, and specialties to travel to Robinson Crusoe to help prevent the extinction of endangered species and support the local community as it works toward achieving sustainability, using newly established internet connectivity and computing infrastructure hosted in a community technology hub established by Lenovo.
On the conservation side we are looking for volunteers with experience in machine learning, artificial intelligence, and data management to advance the first community-led, data driven island restoration project to protect the threatened Pink footed shearwater. This includes supporting the local team in training and implementing camera trap classification models and aggregating detection data and field observations into quantitative analysis of project progress.
Those interested in becoming one of the lucky volunteers to support this project on Robinson Crusoe Island while being able to continue working their current day jobs remotely can find out more and apply at www.LenovoWFH.com by December 30, 2021.
*Thread Reply:* Do you think Lenovo would be open to paying participants for their conservation work? The work Island Conservation is doing is amazing and the conservation goals here are obviously very important - but if Lenovo wants to make a larger impact, it seems to me a great contribution would be to directly invest in the conservationist labor. Especially considering Lenovo has $60B+ in revenue and seems to be getting a good amount of marketing benefit out of this (their name is mentioned 26 times in this article). I also can’t help but feel the study on Gen Z work preferences is not recognizing the anxiety a lot of people face now due to lack of job security in the gig economy - being paid is also a great way to communicate that your work has value! Again, I’m all for the conservation goals, this is more so a comment on big tech getting benefit from free labor… maybe they can be convinced to invest further 🙂
*Thread Reply:* If you have little overlap in time AND frequency with other vocalisations, you can use basic automatic segmentation as a basis for a quick labelling, like in Tadarida software: https://openresearchsoftware.metajnl.com/articles/10.5334/jors.154/ You select groups of pre-segmented calls at once
*Thread Reply:* Sounds like a fun project! Assuming all the audio you'd like to process is of variable lengths, you might find it necessary to split the audio up into discrete pieces. With this in mind, knowing the time of each call would help you decide how best to split up the data.
If your training data is scarce, splitting it up can be useful for augmentations such as layering it over clips of environmental noise. The one catch here is you will have to take additional caution when splitting training and testing data.
Also, might be worth looking into attention modules (i.e., transformers). I've had great success in the past using them for audio processing. Here is a source that helped me: https://arxiv.org/abs/1803.02353
*Thread Reply:* Hey @Jason Holmberg (Wild Me), hope you’re doing great! There’s not a ton of work at exactly the intersection of camera traps + video, but I’d be happy to share what we learned in our latest round of reviewing things and implementing zamba.
*Thread Reply:* Shoot me an email and we can find a time
*Thread Reply:* FYI we have a bit of experience with using MegaDetector on videos... what we do is wildly inelegant (sample frames, run MD, do something sensible to aggregate the results from frames back up to videos), but has been useful to a number of users. Timelapse (which is how most of our users consume MD results) has good support for this, so it didn't require any new UI tooling. So, this is at least an "existing approach". Whether this is "state-of-the-art" is left as an exercise for the reader. 🙂
https://github.com/microsoft/CameraTraps/blob/master/detection/process_video.py
<a href="https://github.com/microsoft/CameraTraps">microsoft/CameraTraps</a>
Data4Wildlife challenge: https://www.bright-tide.co.uk/data4wildlifechallenge
Historically, the top three factors driving the global biodiversity extinction crisis have been habitat loss, human-wildlife conflict, and poaching. Over about the past decade, a new factor has propelled into the spotlight: online crime. Online wildlife crime causes species declines. The algorithmic amplification of wildlife crime via social media platforms, mobile applications and high-speed broadband access is a grave concern for nongovernmental organizations working to combat online crime. Join us for the #DATA4WILDLIFE challenge and help slow species declines! The #DATA4WILDLIFE challenge will take place on 29/30 January 2022. Applications are now open and will close at 22.00 on 12 January 2021. Teams will be announced on 17 January 2022 and drop in mentorship sessions will be held during that week to prepare teams for the two day challenge on 29/30 January 2022.
https://www.nytimes.com/2021/09/30/opinion/animal-extinction.html?smid=tw-share
*Thread Reply:* Also realize that many of you in academic positions have a lot more experience with industry careers and with answering this kind of question for students than I do, so y'all fire away here as well please!
*Thread Reply:* Hey again Carl! Ive been working for industry (mostly R&D groups) for while now, and at various points responsible for interviewing and hiring. Here are some general thoughts, hope they help.
I do not speak for any company in particular, but when evaluating non-researcher MLEs (researchers typically have a mandate very similar to their counterparts in academia), I am typically looking for people with a track record of solving interesting, application-minded problems. A solid grasp of the fundamentals is necessary, but what I usually want to see is evidence that a candidate can apply all that useful theory they learn in schools to actually solve an open ended customer problem. This can be shown with project work, contributing to open source tools, making an impact at a past internships, etc... Degrees I typically look for are the traditional STEM ones for this field (EE, CS, CompE, etc..). Other majors, including math, applied math, physics, other engineering disciplines (e.g. BME), and others arent disqualifiers but sometimes these candidates are weaker on the core tools needed for the job (like programming) and need to demonstrate that experience through prior project work as I mentioned. Non-academic project work also usually implies they have experience with programming languages in the wild, working on a team, and delivering under customer imposed deadlines. Also, it helps not to be an entitled jerk. 🙂 Cant tell you how many students from top programs Ive met who throw their resume in my face and expect a job offer on pedigree alone. Insta-no
*Thread Reply:* If the students are interested in conservation/environmental fields, getting involved now as a volunteer is a great way to build experience specific to how agencies and organizations in that world operate, and even more importantly a great way to build a professional network that is just as important as the resume. The volunteer work doesn't have to be specific to tech, you just want to get involved in whatever the work is (habitat restoration, river cleanups, species protection, clean water, affordable housing, etc.) and then help the orgs figure out how tech can make that work more efficient, scalable, etc.
*Thread Reply:* Maybe recommend Designing your Life as a general resource for designing a nonstandard career path?
*Thread Reply:* As someone who recently graduated from college with some (limited) industry experience, I'd say networking is key. Don't be afraid to cold email someone for an informational interview or chat over tea/coffee/video/phone if they're part of a startup/company working on sustainability and conservation.
Hi everyone!! I'm Kakani Katija, a bioengineer at MBARI (Monterey Bay Aquarium Research Institute) where I lead the Bioinspiration Lab (we do lots of underwater imaging, robotics, bioinspired design). We (along with LOADS of collaborators) recently launched FathomNet, an underwater image database that contains labeled data for ocean life that will hopefully grow with community contributions. Check it out (and the blog and github) and let us know where we can do better. If you're interested in the ocean space, I'm also happy to chat!
Please share this fantastic internship opportunity with anyone you think may be interested https://boards.greenhouse.io/thealleninstitute/jobs/3738739
*Thread Reply:* if any students here want to take on a small project.
*Thread Reply:* @Benjamin Kellenberger Maybe AIDE is a good fit?
*Thread Reply:* I think the Microsoft AI4E camera trap classifier lives on with Dan Morris -- he might be able to help them find which images have any animals
*Thread Reply:* Good opportunity to finally combine MegaDetector (MD) with AIDE? It’s been on my list ever since. Pre-processing with MD and import into AIDE is another option that is almost immediately available. I’d be happy to help out if you find my software to be a potential candidate!
*Thread Reply:* Megadetector or Zamba are good for this. @Sara Beery
*Thread Reply:* @Ben Weinstein you could point them to MegaDetector GUI . It depends if MegaDetector fits their needs in the first place. I haven't made updates to that project in some time but it's in a relatively stable state (I think there are some bugs surrounding metadata export). Feel free to point them to me if they want to discuss things. Cheers.
*Thread Reply:* Cool project @Petar Gyurov! Didn’t know about it before.
*Thread Reply:* @Ben Weinstein MegaDetector may be helpful here, but I agree with Petar that it’s helpful to determine whether MegaDetector fits their needs in terms of accuracy and image distribution, and if it does, we can help them find a workflow to use the results. We try to “fail fast” if MegaDetector isn’t going to work for someone, so feel free to have them email cameratraps@lila.science and we’ll help them figure it out. If you’re interested in following along, fine to to have them cc you as well. Thanks!
*Thread Reply:* Hello Monty, I stumbled about this very recent review (October 21) called "Seeing biodiversity: perspectives in machine learning for wildlife conservation" when I got interested in the same topic. They already did an amazing job and I think it might be of help to you. You can find it here: https://arxiv.org/abs/2110.12951
Merry Christmas!
*Thread Reply:* Merry Christmas! Thanks for this, it looks like a fantastic review. Will definitely be useful for me!
*Thread Reply:* Though it’s specific to AI for camera traps, you may find the list of papers (most with one-paragraph summaries) here to be useful:
https://agentmorris.github.io/camera-trap-ml-survey/#camera-trap-ml-papers
*Thread Reply:* This is vastly useful, thanks Dan 👌
*Thread Reply:* ^also open to the broader community beyond PhD students, including undergrad, master's, and postdocs as well as folks outside academia!
@Jake Wall and I brainstormed this joint local/international conservation technology internship at the Mara Elephant Project while we were in the field last August, and I'm so excited to see it come to fruition thanks to Ai2! If you have a student who might be interested, please pass the information along! https://boards.greenhouse.io/thealleninstitute/jobs/3820329
*Thread Reply:* @Bistra Dilkina you were looking for undergraduate opportunities
*Thread Reply:* Whoa!! What a cool opportunity!
*Thread Reply:* Kudos as well to @Jes Lefcourt for making this happen!
*Thread Reply:* What are the start dates? And is there an application deadline? Is it paid/are expenses like flight/lodging covered?
*Thread Reply:* @Ted Schmitt can probably best answer re: the logistical details from the Ai2 side
*Thread Reply:* Ok, asked about it on twitter too & didn’t get a reply. Seems like some important details…
*Thread Reply:* I agree! I just don't know the answers :)
*Thread Reply:* Though I'm 99.9% sure that it's paid and travel is covered
*Thread Reply:* Yeah I figured it would be given the funders behind it. I’ll wait for Ted’s response. Thanks Sara!
*Thread Reply:* Sorry, I was double-confirming the details. The start date is flexible, and all expenses are paid.
*Thread Reply:* Ok great, thanks for the clarification Jes!
*Thread Reply:* i know the start date is flexible, but is there still an application deadline?
*Thread Reply:* Hey @Carly Batist, I work on torchgeo 🙂, please don't hesitate to reach out if you have any questions, comments, or something is broken!
*Thread Reply:* Ah so sorry I didn’t tag you in the original post😬, didn’t realize you were already here! Saw it on Twitter and thought it would super useful
💥New year, new update to the Conservation Tech Directory! 577 entries now🥳🤩. @Gracie Ermi https://conservationtech.directory/
Reminder - now with academic lab as resource type. If you’ve got a #ConservationTech lab, please add it! (link below) Google form to add new resources. And updated static doc/PDF on figshare.
A wonderful internship opportunity at the Allen Institute to work on their Mara Elephant Listening Project
https://boards.greenhouse.io/thealleninstitute/jobs/3820329
I am getting conflicting information on whether how/whether we can download google satellite images and use them in publications/training models. Maybe @Sara Beery and can get me an answer here. I see two different licenses with opposite interpretations. I saw this (https://www.mdpi.com/2072-4292/14/3/476) today which uses our lab's data + some google earth they scraped at 11cm.
*Thread Reply:* Google seems to have really complex policies, I can try to ask.
*Thread Reply:* Aside: Another imagery option is to request the ESA for data from the Pleiades satellites: https://www.intelligence-airbusds.com/imagery/constellation/pleiades/. The satellites offer high resolution imagery (50cm/px iirc and a 26 day revisit cycle). The data is free for researchers if accessing areas within a certain size limit.
*Thread Reply:* thanks @aruna!, this is actually on my to do list, I know it went up last year, we have been purchasing from MAXAR. Do you have any experience with these data? For example, PLANET claims 50cm data, but it turns out that is a bit of a stretch, its 65cm data sharpened to 50cm through interpolation. I had a long discussion with their technical team.
*Thread Reply:* Hi Ben! No, unfortunately I don't have any experience with the actual data itself. Definitely sounds like you have spent more time with the team. have you had a chance to look at the actual data?
*Thread Reply:* More info: ```FGVC9 - The Ninth Workshop on Fine-Grained Visual Categorization 19th June 2022 @ CVPR 2022 Website: https://sites.google.com/view/fgvc9 Twitter: @fgvcworkshop Email: fgvcworkshop@googlegroups.com
This workshop brings together researchers to explore visual recognition across the continuum between basic level categorization (object recognition) and identification of individuals (face recognition, biometrics). Participants are encouraged to submit short papers and to take part in a set of competitions organized in conjunction with the workshop - details below. We will also have an exciting lineup of invited speakers from computer vision through to domain experts.
PAPER SUBMISSION We invite submission of 4 page (excluding references) extended abstracts on topics related to fine-grained recognition. Reviewing of abstract submissions will be double-blind. The purpose of this workshop is not specifically as a venue for publication so much as a place to gather together those in the community working on or interested in FGVC. The workshop proceedings will not appear in the official CVPR 2022 workshop proceedings. Submissions of work which has been previously published, including papers accepted to the main CVPR 2022 conference are allowed.
PAPER SUBMISSION DATES * Deadline for Submission - 25th March 2022 * Notification of Acceptance - 25th April 2022 * Camera Ready - 6th May 2022 * Submission will be via CMT
Scope The purpose of this workshop is to bring together researchers to explore visual recognition across the continuum between basic level categorization and identification of individuals within a category population. Topics of interest include:
Fine-grained categorization Novel datasets and data collection strategies for fine-grained categorization Low/few shot learning Self-supervised learning Semi-supervised learning Transfer-learning Attribute and part based approaches Taxonomic prediction Long-tailed learning
Human-in-the-loop Fine-grained categorization with humans in the loop Embedding human experts' knowledge into computational models Machine teaching Interpretable fine-grained models
Multi-modal learning Using audio and video data Using meta data e.g. geographical priors Learning shape
Fine-grained applications Product recognition Animal biometrics and camera traps Museum collections Agricultural Medical Fashion
COMPETITIONS We will also be hosting several fine-grained computer vision challenges with tasks ranging from classification of attributes in art images through to classifying diseases in plants. The competitions will be hosted on Kaggle and will be announced in late February 2022.
ORGANIZERS Sara Beery - Caltech Serge Belongie - Cornell Elijah Cole - Caltech Xiangteng He - Peking University Christine Kaeser-Chen - DeepMind Oisin Mac Aodha - University of Edinburgh Subhransu Maji - University of Massachusetts, Amherst Abby Stylianou - Saint Louis University Jong-Chyi Su - University of Massachusetts, Amherst Grant Van Horn - Cornell Kimberly Wilber - Google```
*Thread Reply:* Have you seen the iNaturalist API: https://api.inaturalist.org/v1/docs/#!/Taxa/get_taxa
e.g. https://api.inaturalist.org/v1/taxa?q=european%20robin returns a json file with the species name
*Thread Reply:* We use the ITIS API
*Thread Reply:* Not sure if it multilingual
*Thread Reply:* Just keep in mind that common and scientific names frequently change meanings over time as the taxonomy is revised. Same name != same concept just like when you have to align multiple ontologies
*Thread Reply:* We had to solve this problem to a slightly weaker standard than others may have: we wanted consistent mapping to a common taxonomy for lots and lots of records from all over the world, but because we weren't publishing the results, it didn't have to be "right" in every single case, just "right enough" and consistent, and we had to have a way of quickly making manual corrections. E.g. we had to have a way to permanently indicate that "lion" in our Idaho data meant something different from "lion" in our Snapshot Safari data... as you can imagine, there are 1000 special cases like that. We also needed a way to quickly verify the results by looking at images, since I don't actually know the scientific names for anything, and if you're not careful, you will map "wolf" to a fish (yes, there is a fish called "wolf"). So, with a huge dose of "YMMV", we ended up with the code here:
https://github.com/microsoft/CameraTraps/tree/main/taxonomy_mapping
...mostly here:
https://github.com/microsoft/CameraTraps/blob/main/taxonomy_mapping/species_lookup.py
...which basically downloads both the iNat and GBIF taxonomy files, maps lots of common-name queries into both taxonomies (preferring iNat), and makes a big spreadsheet that makes it easy (or at least minimally painful) to double-check all the mappings and manually fill in things that failed to map to anything.
I'm not sure I would advocate anyone actually running that code, but I would definitely advocate for the overall approach, which worked well and got the manual work of mapping naming schemes for zillions of images down to maybe an hour.
<a href="https://github.com/microsoft/CameraTraps">microsoft/CameraTraps</a>
Could be interesting for some of you, a special issue from Frontiers on ML/DL for ecological monitoring : https://www.frontiersin.org/research-topics/32067/advances-in-machine-learning-and-deep-learning-for-monitoring-terrestrial-ecosystems. The deadline for the abstract is March 27, and the manuscript deadline is May 26.
This upcoming (virtual) conference could be of interest to anyone working on or interested in applying ML to the digitization of biodiversity museum collections: https://www.idigbio.org/content/digital-data-2022-enhancing-advancing-quality-digitized-data
*Thread Reply:* There are lots of great wildlife datasets on https://lila.science/
plus the iWildCam competition datasets https://github.com/visipedia/iwildcam_comp
And the iNaturalist datasets: https://github.com/visipedia/inat_comp
*Thread Reply:* @Björn Lütjens does this relate to https://arxiv.org/abs/2112.00570, I was literally just emailing Alex and ccing @Sara Beery. We are getting some data out the door and maybe into torchgeo (@Caleb Robinson) spoke yesterday.
*Thread Reply:* Some of the datasets included in the climate change benchmark are also listed in @Björn Lütjens’ repository but I believe the benchmark plans to be more than that (a full fledged environment to train and validate self-supervised algorithms). Alex mentioned it would be super cool to integrate your data @Ben Weinstein !! 📡
*Thread Reply:* yup. We are actively talking about it now.
*Thread Reply:* @Chris Yeh Some of the SustainBench dataset (s) might also be relevant?
*Thread Reply:* Amazing thanks!!! I added pointers to lila.science, iWildCam, iNaturalist, and SustainBench. LILA has two cool datasets on forest health and canopy height. I couldn't find directly relevant datasets on SustainBench, but provided a pointer. Thanks!!
Lmk, in case you hear of any good wildfire or belowground carbon datasets 🙂
*Thread Reply:* There is a wildlife/wildfire intersection dataset on Wildlife Insights, from Australia, looking at recovery after brushfires. But I don't think it's ML ready. https://www.worldwildlife.org/stories/an-eye-on-recovery
*Thread Reply:* very cool! I couldn't find a link to the dataset, they might still be processing it. But I'll keep an eye out for this data, thx!
*Thread Reply:* I think @Carly Batist and @Gracie Ermi also have datasets listed in their Conservation Tech directory! https://conservationtech.directory/
*Thread Reply:* > @Chris Yeh Some of the SustainBench dataset (s) might also be relevant? Re: @Sara Beery - none of the SustainBench datasets deal with wildlife, wildfire, trees, or carbon. We decided against wildlife because WILDS / iWildcam + lila + iNat already cover wildlife quite well, and we didn't have any in-house (Stanford SustainLab) expertise on the others. (We searched for trees / carbon datasets, but we didn't find any that seemed ready-to-go for inclusion. Maybe in a future version tho!)
*Thread Reply:* Thank you so much for the thoughts @Lily Xu and @Chris Yeh . I'll add a link to conservationtech.directory
*Thread Reply:* @Björn Lütjens We’ll add this forest dataset repo to the directory as well in our next update!
Hi all, we have published recently a report on some work we have been doing on fish sounds, I would be interested in some feedback and suggestions! https://arxiv.org/abs/2201.05013v2
*Thread Reply:* y'all are doing such a valuable thing thank youuuuu
*Thread Reply:* That’s an amazing list on your Web page indeed! I second @Lily Xu’s comment; thanks a lot for your work!
Regarding contents in the screenshot: I just noticed TensorFlow, which begs the question on how generic you allow entries to be. In that case you could open the box of Pandora with Python and add PyTorch, SciPy, Scikit-Image, sklearn, etc.
Otherwise I’ll just speak for a colleague and mention a must-have: DeepLabCut: https://github.com/DeepLabCut/DeepLabCut
*Thread Reply:* We actually just took TensorFlow (& some others like seewave) out because of being too ‘general’, we were debating where to draw the line and it gets hard! But we agree, thanks for your input!
*Thread Reply:* Mountain Lion doesn’t seem to be on the list, but the go-to experts in animal re-ID for me are the folks at WildMe: https://www.wildme.org/#/ @Jason Parham @Jason Holmberg (Wild Me)
*Thread Reply:* We currently don’t have a Mountain Lion re-ID project, but we in discussion about starting a North America carnivore platform
*Thread Reply:* We have never seen the baseline dataset of known IDs that would be the starting point of a future effort
*Thread Reply:* Agreed, @Alayna Van Dervort do you have a current Mountain Lion photo ID dataset?
*Thread Reply:* I love this idea. Are you in touch with Beth Pratt (NWF, P-22 advocate) or Liz Hadly at Stanford?
*Thread Reply:* @Jason Holmberg (Wild Me)
*Thread Reply:* Hi @Jason Holmberg (Wild Me) I will email you directly !
*Thread Reply:* Hi Jason! We can absolutely add that term to your description so that if someone searches ‘docker’ WildMe will still show up, but we try to keep the tags to more generalized topic areas so that we don’t end up with dozens of them. Hope this is an ok solution!
*Thread Reply:* https://www.epfl.ch/about/working/phd-position-mapping-wildlife-environment-interactions-in-the-swiss-alps-with-ai-wildai/ this is one
*Thread Reply:* https://www.epfl.ch/about/working/phd-position-target-shift-in-species-distribution-modeling/ and this is the other!
*Thread Reply:* I've got a project on Azure right now and would love to be added to this Slack channel! Thanks Sicily 🙂
*Thread Reply:* I have had my fair share with Azure too, also through my software, and would like to contribute to, and learn from, this channel, too! Aside that, we’re also working on (aerial) bird detection.
*Thread Reply:* Hi both! Could you please share your emails via PM
*Thread Reply:* Hi Sicily 👋 could you add me as well? I'm not a PhD student but do use Azure!
*Thread Reply:* Hello, Yes please I would love to be added, av@thebigwild.com
*Thread Reply:* Summary: Postdoc in Machine Learning for Large Scale Biodiversity Mapping at the University of Edinburgh Duration: One year, starting from April 2022 (can be negotiated) More information and how to apply: https://edin.ac/3hutyUG Additional queries: Contact Oisin Mac Aodha - https://homepages.inf.ed.ac.uk/omacaod/
Details: The School of Informatics, University of Edinburgh invites applications for a research associate in the area of machine learning for large scale biodiversity mapping. The project is part of an exciting collaboration between the School of Informatics and the citizen science platform iNaturalist, and features input from the International Union for Conservation of Nature (IUCN).
Species range maps, describing where species do and do not occur, are critical to prioritizing scarce conservation resources in order to mitigate some of the worst potential impacts of climate change on biodiversity. Citizen science platforms such as iNaturalist generate millions of casual observations each year which contain information about where thousands of different species have been observed. However, constructing accurate species range maps is a time consuming and laborious process that is further hampered by data quality and coverage issues.
This project aims to develop novel machine learning-powered methods for producing plausible species range maps from citizen science data. The developed methods will enable the joint modelling of thousands of different species' distributions. These methods will make use of, and advance, the state of the art in deep learning-based density estimation in the context of noisy geospatial data.
For additional information please check our previous related project: https://edin.ac/3vvFyh2.
https://www.nytimes.com/interactive/2022/03/03/climate/biodiversity-map.html
*Thread Reply:* @Laure Delisle what type of mines were you looking at?
*Thread Reply:* You might ask the folks from the ConX artisinal mining challenge? I think some of the solutions were working on remote sensing of mines. https://www.artisanalminingchallenge.com/previous-round
*Thread Reply:* @Carly Batist Thank you, i’m already in contact with them 🙂. While waiting to hear more information, I thought to come here and see if there are any platforms containing them that I’m missing out on.
Gentle reminder, one week left to apply for PhD positions in AI for conservation in beautiful Switzerland! https://www.epfl.ch/labs/eceo/eceo/open-positions/
Hi everyone! I work at Edge Impulse; we create tools to help developers build machine learning models for deployment to embedded devices. We're not a conservation tech company, but we do a ton of conservation-related work:
• We fund and mentor the WILDLABS Fellowship: On the edge • We organize community conservation tech efforts, like ElephantEdge • We work very closely with some amazing conservation tech organizations, like Arribada and CXL • We are regular speakers at community events like WILDLABS tech tutors • We donate 1% of our revenue to environmental nonprofits I'm posting here because we're looking for a machine learning engineer to join our team. You'd spend your time creating new technologies around machine learning on edge devices—including those that have the potential to directly benefit conservation research. For example, one big project we're working on right now is training embedding models for camera traps. If this type of thing sounds appealing, drop me an email at dan@edgeimpulse.com. Thank you!!
Hi all! We’re organizing the 2nd CV4Animals workshop at CVPR 2022! We’re inviting abstracts to present at the workshop (either by poster or oral), deadline April 29th, please see the website and attached PDF for more details! https://www.cv4animals.com/home
💥Conservation Tech Directory update!💥
New feature - we’ve included conservation tech stats packages (python & R/rstudio tags). And now at 618 resources!🤩. We’ve also created a new resource type (Tool) & re-classified resources to this. Includes datasets, toolkits, software packages, etc. https://conservationtech.directory/ PDF version: https://doi.org/10.6084/m9.figshare.15442200. @Gracie Ermi
*Thread Reply:* @Akronix Just use the zoom link to log in
*Thread Reply:* Unfortunately I can’t make it on time. Will it be recorded?
*Thread Reply:* Yes, the talks are recorded. We will upload recordings to YouTube
*Thread Reply:* Could you send me the link once available on YouTube? Thanks!
*Thread Reply:* For urban tree detection + species classification, there was the registree project led by Caltech and ETH (http://www.vision.caltech.edu/registree/)
*Thread Reply:* (but you should know it, actually 😉, now that i think about it )
*Thread Reply:* For tree crown segmentation, look at works by Antonio Ferraz or Martin Weinmann. They have been doing a lot in that direction
*Thread Reply:* Thanks yes that’s definitely one option too! 🙂
*Thread Reply:* You should think of DeepForest as a baseline model that you build from, I've heard a number of people add in urban tree detections. Just spend 2 or 3 days annotating and finetune and you should be in good shape.
*Thread Reply:* i'm happy to host a urban tree model branch if gets up and running.
*Thread Reply:* Great thanks!
https://coomeslab.org/dr-thomas-swinfield/ - Tom swinfield has been working on this for the last few years, but it's in tropic forests rather than urban landscapes. He /his research group might have some pointers
*Thread Reply:* Thanks, I’ll look into that!
*Thread Reply:* I also wonder if some of urban tree mapping citsci platforms have worked on something
*Thread Reply:* Hi Alexander, you should check out David Dao! He’s also doing a lot of work in carbon credit estimation and secure payment for smallholder farms on the carbon markets in the Global South, specifically Central and South America
*Thread Reply:* https://earthshot.eco/
*Thread Reply:* Also check out Regen Network https://www.regen.network/
Happy to announce iWildCam 2022, our 5th annual camera trap challenge, focused on helping ecologists monitor biodiversity. Join the Kaggle competition and help us count individual animals across sequences of images.
https://www.kaggle.com/c/iwildcam2022-fgvc9
[Competition co-organized with @Sara Beery & @John Beuving, with valuable support from @Dan Morris.]
*Thread Reply:* Hi Stefan,The competitions are very exciting. However, as a non participant, I always have this question in mind about the IWILDCAM competition: besides knowing which team wins the competition, how can we actually know how this team proceed (algorithms, software, and so on)? In the open science era, it would be great to know... In any case, thanks so much for organizing this kind of event.
*Thread Reply:* Vincent, in the discussion board for each competition the winning teams post their solutions! You can go to any of the past competitions and read about the winning solutions. We also have recorded videos from the last two workshops of a discussion of the competition-winning results for all the FGVC competitions
*Thread Reply:* We're hoping to write up a larger "what we've learned" paper covering the past competitions and the themes of what does and doesn't work. Kaggle can't enforce teams publishing their code unfortunately (unlike LifeCLEF!) but some competition winners have published their code independently (like last year's winner here: https://github.com/alcunha/iwildcam2021ufam)
*Thread Reply:* Competitions like this don't end with a deadline and a winner. Oftentimes participants (are encouraged to) share their solutions afterwards, in forums, papers or workshops. I recommend to everyone interested in learning more to revisit such competitions after a while, and I am sure new materials will become available.
For example, for iWildCam 2021 there was a FGVC workshop afterwards: https://www.youtube.com/watch?v=yKP_tX_gTk0, and participants shared full solutions (as Sara mentioned), or just descriptions of their solutions on the Kaggle forums:
1st place: https://www.kaggle.com/c/iwildcam2021-fgvc8/discussion/245460 2nd place: https://www.kaggle.com/c/iwildcam2021-fgvc8/discussion/245559 3rd place: https://www.kaggle.com/c/iwildcam2021-fgvc8/discussion/244950
*Thread Reply:* Thanks for these precise answers. I missed these information, sorry about that. And, again, thanks for organizing these competitions 👍
If anyone is doing any work around fisheries in the US or Canada, take a look at this forgivable loan program from Multiplier. Deadline April 15. https://multiplier.org/2022/03/announcing-the-technology-adoption-fund-for-sustainable-fisheries-and-the-inaugural-round-of-loan-funding/
👋 everyone - I’m Fernando Pérez from UC Berkeley, and wanted to introduce myself as well as let you know of a new program we are just launching that I hope will be a great point of connection with this community: the Eric and Wendy Schmidt Center for Data Science and Environment at Berkeley, or ‘DS4E’ for short (our website is here).
The short version is that: • Our mission is impact-oriented, not academic research. We want to combine the expertise of environmental scientists, engagement with stakeholders close to the issue, and expertise in software engineering and data science, towards concrete, implementable, deployed tools that can contribute to real-world problems. • Our specific topics to focus our effort into are not yet pre-defined, and instead we will organize later this year (once we have some staff hired!) events to consult with the community how to best allocate our resources and work together for maximal impact. The above is only a launch press release, as we are currently in early stages of remodelling space, getting job ads posted, etc. But we wanted to connect with you all in the hope that you will be interested in opportunities to collaborate with us, apply for positions, participate in future events, etc.
I wanted to particularly thank @Sara Beery for kindly welcoming us to this space - we hope to meet many more of you over time!
From our launch team, both @Carl Boettiger and I are on this Slack and will be happy to answer any questions you might have. Thanks for reading!
*Thread Reply:* So cool!! I’ve added it in to our next update of the Conservation Tech Directory!
*Thread Reply:* Very exciting and interesting project! Looking forward to see the vacancies for new members of the team!
*Thread Reply:* Wonderful news and congrats on the launch! An inspiring mission for all of us passionate about impact-oriented work
*Thread Reply:* @Kakani Katija sounds similar to some of the FathomNet data?
*Thread Reply:* We maintain an open source DIY-AI tool for NOAA, called VIAME. Originally targeting fisheries but we've since expanded to a number of subsea applications. It supports a number of workflows such as detection, tracking, counting, etc... while also providing a means of labeling the data. The idea is for a scientists or decision maker to be able to train models without knowing anything about ml, or even code. Just bring the SME perspective and some time to label haha. It's been deployed at many of noaa's sites already and some of our collaborators at various uni and commercial companies have found it useful. It's completely free and open source, we dont make products or hold any IP at kitware. Please feel free to reach out if you'd like me to connect you to our program manager or engineering team.
*Thread Reply:* https://github.com/Kitware/dive
*Thread Reply:* Yes, there's also the Tator annotation tool by CVision AI that has AI tooling as well: tator.io. Web-based and easy to get started.
*Thread Reply:* I made a tiny GUI a few years ago 'motion detection only' for finding periods of activity. Nothing like what @Daniel Davila is doing in terms of sophistication. http://benweinstein.weebly.com/deepmeerkat.html
*Thread Reply:* In case you want a full blown underwater camera trap (definitely overkill if you have access to continuous power), I worked on one for manta ray research in 2013 that was more for deeper reefs (~30m) : https://ando.xyz/work/manta-id
*Thread Reply:* Hi Danielle, I am not sure if it is applicable for your study, but we published a bounding box annotated underwater dataset a few years ago in murky waters. Maybe it can be used as training data for your system 🙂 https://www.kaggle.com/datasets/aalborguniversity/brackish-dataset
*Thread Reply:* Wow thank you so much everyone for your input and help! I have some more options to explore and see what works 🙂 Didn't think what I was doing was too novel and glad to see that it isn't
*Thread Reply:* @Danielle Montocchio If you're interested, we're having a workshop Thursday/Friday morning (Pacific) (starts in ~20 minutes) that walks people through resources like Tator and FathomNet. Registration link is here: www.tinyurl.com/fathomnet
Some exciting improvements for iWildCam 2022!!!
https://twitter.com/sarameghanbeery/status/1512102482641444868
}
Twitter
*Thread Reply:* I hadn't realized this was the focus. This is awesome!
*Thread Reply:* People ask us relatively often: "can I just count MD boxes above some threshold and use that to estimate the # of animals in an image?" We basically always say "no we don't recommend that", but I actually have no idea what the error distribution looks like, we just don't want to assume that it does anything remotely useful for counting. I know this competition is focused on sequence-level counting, but surely some enterprising competitor will make a baseline entry that just takes the per-image MD results that are on the competition page and the GT counts for the training data, and see what the numbers look like? And maybe that enterprising competitor will post a summary here? :)
*Thread Reply:* We might even put that on the leaderboard as a baseline (we did last year, but it had the complexity of species ID tied in) 🙂
*Thread Reply:* Yes, do that! I recommend the following advanced algorithm: { count = max(# of boxes) }
🚨Conservation Tech Directory update!🚨
Lots more entries, but biggest update is that we now have a listserv you can subscribe to if you want direct notifications in your inbox of future directory updates! So we don’t have to keep spam’ing you on social media😅 Through the site: https://conservationtech.directory/ Directly via sign-up form: http://eepurl.com/hWwV0X
@Gracie Ermi
Now 55000 taxa in Inaturalist computer vision model: https://www.inaturalist.org/blog/63931-the-latest-computer-vision-model-updates
Sound ID 2.0 is now out for iOS devices! Android will be available next month. While the UI looks the same for 2.0, everything "under the hood" has been rewritten and optimized: • efficiency is 10 to 100x better, depending on iPhone version ◦ You should be able to start Sound ID and leave it running while on a walk, hike, bike ride, etc. Get out and explore! • precision and recall are up across the board for all species • we now cover 560 US/Canada species, and 250 Western Paleartic species ◦ Lots of additional species on the horizon! Hope you all enjoy! https://merlin.allaboutbirds.org/
*Thread Reply:* Amazing. My wife and I have loved using this app while on walks and hikes. Thanks to everyone who's worked hard on this, both app and model-wise. Cool cool stuff
*Thread Reply:* Totally! I was using it extensively just this weekend!
*Thread Reply:* amazing. What was the main innovations that increased accuracy? Just more data?
*Thread Reply:* More data, smarter loss functions, smarter and more aggressive augmentation functions. We have a pretty good grip on annotating audio now and its allowed us to move way past weakly supervised methods.
*Thread Reply:* (we are still at weakly supervised methods for trees). Good to hear.
*Thread Reply:* We also have a 1 week active learning cycle in place, so the annotators have a much better grip on where effort is helping, and where effort needs to be spent
*Thread Reply:* the models' ability to ID even with lots of background noise is pretty impressive
*Thread Reply:* how do you find annotators? If I can be useful for connecting anyone in ecuador/columbia, let me know.
*Thread Reply:* Awesome, I'll keep that in mind! Currently we just tap into folk we know from prior experience. We've built out tool ecosystem around maximizing the efforts of a few experts instead of tapping into the big citizen science community. At some point we'll probably try to embrace some of that energy.
*Thread Reply:* @Declan Thank you! Yeah, that can be attributed to our more aggressive background augmentation strategies and to the detailed annotation protocol.
*Thread Reply:* I'm looking forward to the Android update. The app really helps my birding. When I can't tell where a call or song is coming from, the Merlin ID helps me know where to look!
*Thread Reply:* this app is great! ......... would love to have a "mic trap" mode (is that a thing? like camera-trap but for audio?) where i could set this thing up outside and see what birds it picks up over the timeline of the day. 😄
*Thread Reply:* Grab some Audiomoths 🙂
*Thread Reply:* Just extensively tested this morning. The new version is really quite impressive. Huge congratulations. I love the highlight spectrogram and compare to call feature. I feel like this is a new high water mark for ecology machine learning and integration with existing tech.
*Thread Reply:* Correctly caught a single western sandpiper in a flock of sanderlings from 40 feet amongst the heavy surf.
*Thread Reply:* Thanks for spreading the word Priya! Happy to answer any questions (I am the contact person in the RFP)
New season of WILDLABS virtual meetups centered around movement ecology!
https://twitter.com/WILDLABSNET/status/1517065606415142913
}
Twitter
https://www.eventbrite.co.uk/e/virtual-meetup-data-analysis-in-movement-ecology-tickets-328359842127
*Thread Reply:* "Leopards and hyenas and belugas. Oh my!"
*Thread Reply:* I learned about this competition from the beluga data set:
https://www.drivendata.org/competitions/96/beluga-whales/
DrivenData truly has the market cornered on conservation puns. See:
https://www.drivendata.org/competitions/59/camera-trap-serengeti/
*Thread Reply:* I think that was physical pain I just felt with "Hakuna ma-data".
*Thread Reply:* @Dan Morris “Whale Known Belugas”, “Belugas Got Back”, and “Fluke Box Hero” were also floated internally before the competition launched. One of the hardest things we had to do to get the competition set up was deciding on the best pun.
*Thread Reply:* I'll make you a deal... if you do another competition next year, and you call it Fluke Box Hero, I will personally record the parody theme song.
*Thread Reply:* Shoutout to @Greg Lipstein and the rest of the DataDriven team for making that decision really hard.
*Thread Reply:* Welcome and congrats for the well designed TrapTagger!
*Thread Reply:* Awesome!! Will put this on our To-Add list for the next Cons Tech Directory update
*Thread Reply:* Thanks, it's much appreciated!
Just bringing this conversation to the wider community. I am writing a tree species classification paper (120 species, full continent scale). The question from an ecology collaborator on reading the draft was that our accuracy of about 65% is lower than many machine learning papers in ecology. I'm copying my response below and welcome discussion from others.
"One of my ongoing concerns for publication and presentations is the overall low accuracy rates. I have not done a complete survey of the literature by any means, but it seems to me that machine learning prediction rates are around 70%. Just wondering what your thoughts on if and how to deal with that."
I think your point is true, but indicative of a problem rather than a benefit. There is a pernicious bias in the ecological machine learning literature to 'solving' problems. It is a bit of paradox because every introduction wants to set an applied problem up as being crucial, but every result section wants to show high accuracy values. The result is that there is a real epidemic of lax evaluation criteria. Either through a lack of data, or through weak rigor in setting up the problem that actually reflects the full breadth of the difficulty during prediction. The above figure is a perfect example of that. It is completely natural to do a random train/test split for each class and assess the accuracy per species. If you only had NEON data, you would be deceived into thinking that your accuracy when it comes time to make prediction on the full site scale is more than 150% higher than it actually is. The funny thing is that these problems seem to manifest mostly in the applied papers and analysis. Pure computer vision research that focuses solely on a specific theory doesn't have the pretense of solving any applied problem, and often has low evaluation scores because most problems that are worth solving 1) need lots of data, 2) are genuinely hard. For example, the famous mask-rcnn paper, which has nearly 18,000 citations in 3 years. The first analysis has an average precision of less than 0.4 (https://arxiv.org/abs/1703.06870). As I put together and outline and figures for the paper, my goal is to articulate that tree species prediction, when framed from the perspective of wanting to make predictions on the scale of tens of millions of trees, is vastly harder than the literature would have you believe. The ugly consequence of this dynamic is that applied ecologists become numb and distrusting to results in these papers, since every paper purports to have outstanding accuracy scores. For example in the tree crown delineation task, literature has been reporting greater than 85% accuracy for nearly 15 years, which really begs the question, why would people keep writing papers about such an easy and already solved problem? The only real conclusion is that the analysis as constructed doesn't reflect the actual needs of the applied workflow, but rather makes assumptions either due to a lack of data, or the desire to have higher values. We can show that our model is 2.5x better than a standard off the shelf approach, I think is the crux of it.
*Thread Reply:* Mic drop! Really well-written. I think there are really two issues here: (a) artificial inflation of accuracy numbers via (IMO usually unintentional) questionable evaluations (which you mention), and (b) an assumed coupling of "accurate" and "useful". Addressing (b) likely requires us to start incorporating more human subjects studies, and also some less-quantitative, more anecdotal case studies about whether/how/why AI systems are actually getting used/abandoned. Exciting that some of those have started to roll out in the last couple of years.
*Thread Reply:* "machine learning prediction rates are around 70%" - is this for a specific, well studied task or benchmark? It should be noted that you cant generalize performance like that across datasets. A reasonable score on e.g. coco isnt necessarily a reasonable score on crowdhuman. And when you talk about the constraints of most real world problems, namely that there is usually a significant lack of relevant data, you can usually take SOTA and chop it in half for out of the box performance (without significant engineering around the ML model itself to boost performance)
*Thread Reply:* That being said, what's right here may be beside the point of whether or not you are queuing up an uphill battle with reviewers. If the magic number for a "good" paper is 70%, or 85%, or whatever, that's as good a reason as any to get rejected these days
*Thread Reply:* @Ben Weinstein Thanks for raising this issue, definitely resonates.
I think it's worth pointing out how this illustrates a very stark difference between ecology literature and ML literature regarding benchmarks. I believe these two fields each sit at relatively opposite extremes in this regard, while the ideal is probably somewhere in the middle.
The reviewer makes no reference to a specific benchmark (or apparently even any examples) in quoting 70%. In doing so, the reviewer is quite consistent with how we publish methods in ecology - everyone evaluates their method wrt their own data rather than a shared benchmark.
In contrast, I gather most ML venues would never consider a methodological advance in accuracy that was not tested against some existing benchmark datasets. No one cares about your ability to discriminate cats and dogs in just any collection of cats and dogs photographs, it has to be ImageNet. (experts here can fill in the blanks better, sorry -- there was an excellent thread in #random a while back on this I think).
I think we can all agree both extremes are not particularly productive -- at the end of the day we're trying to solve an abstract problem, not over-fit a benchmark. But for all that we love to celebrate the diversity and system-specificity of ecology data, we can often be too loose in what we consider a comparable task: should we really consider accuracy of ML identifying trees from space is essentially comparable to the performance we expect from other ML literature in say, identifying mammals from photographs?
*Thread Reply:* Nice discussion. Sometimes there is need for a study that identifies specific challenges that algorithms face and labels datasets in ways that allow you to benchmark against those challenges. You will find two examples https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5975165|here and here. If you guys are interested in doing this for tree species identification I will be happy to have a discussion with you to help bring clarity to this area. One more suggestion: iNaturalist is a natural benchmark and you should be able to use the iNaturalist API to benchmark your algorithm against iNaturalist on the iNaturalist dataset. @gvanhorn will be able to tell you if this is feasible.
*Thread Reply:* sorry for coming to this so late.. thanks for raising this topic and for the excellent well rounded argument.. the origins of this sentiment seem to be the trickle down effect of the publication bias in journals for so called "striking" work expected by editors. As for evaluation, i think it's crucial to test the models on "out of distribution" data. we all develop these models in the future hope of them being applied in real life usage, while the standard train-test splits can be far away from what the model will actually see.. it can be hard to get relevant out of distribution data but it gives a more honest expectation of how the model is expected to react to real life.
Starting in one hour!
https://twitter.com/WILDLABSNET/status/1523690388581920769
Twitter
*Thread Reply:* Too bad I can't join, is there a recording later on?
*Thread Reply:* Yes! They always post recordings after, I'll share the link when it's up!
*Thread Reply:* Hi! I'm a PhD student at CUNY and love that conference! It would be a great workshop opportunity for sure
*Thread Reply:* Especially if the forest is broadleaf it is very difficult to make single trees segmentation from images. We were working on that in the past (segmenting single trees from images or Lidar scans) and lidar was very very useful
*Thread Reply:* even by eyes often you don’t see where a tree finishes and where the next one starts and forest is inherently 3D. It’s a though problem, there are a lot of papers in journals like Remote Sensing of Environment and entire confs like silvilaser or forestsat
*Thread Reply:* Yes, it is very difficult to even tell apart one tree from the other by human eyes (I couldn't do it for the life of me). Looking for ways to find a solution. I am not looking for over 90% accuracy, anything over 70% would be good. Do not have high hopes with Mask Rcnn on crowded trees.
*Thread Reply:* One idea would be to bootstrap a larger training set with someone else's model. Have you looked into some of the open source tree inventory models out there? TreeTect comes to mind, there are a few others too.
https://github.com/krakchris/TreeTect
Is the goal to discern individual trees or just segment out the region in the image that is or isnt tree?
*Thread Reply:* I can't claim to be an expert because data vary so widely, but we have spent several years on the problem. A couple observations. 1) Because there is massive intra-class variance inside of 'tree', you need alot of training data, especially if you want to generalize to large forests, or across geographies. Pretraining is key (https://www.sciencedirect.com/science/article/pii/S157495412030011X). 2) Really consider if you need semantic segmentation through mask rcnn or if bounding boxes are sufficient. We have found that the extra effort for training a pixel level model is not worth it. Many trees can be approximated by 4 point boxes and that does not inhibit downstream analysis. It allows you to annotate many more images faster. We use torchvision's retinanet and have released a pretrained model that I'd love you to try. Finetuning it to your data usually works well. https://deepforest.readthedocs.io/. 3) This one is a bit controversial, but outside of using the unsupervised LiDAR algorithms for pretraining in the RGB, i have never seen a LiDAR algorithm that outperforms RGB alone. The caveat here is that i think it depends alot on LiDAR point density. The majority of LiDAR papers use hundreds of points per meter through intense data collection of a small area. When you want to monitor millions of trees, this level of data collection is usually not possible. NEON LiDAR data averages about 4 points per meter over large areas. At this scale, the LiDAR is not that useful, and a canopy height model is often helpful. I do think any multi-spectral data may be useful, we have shown it helps in tropical forests (https://ieeexplore.ieee.org/abstract/document/9387530). 4). If I had to put money on it, the future is in massive pretrained models, after significant research on scaling across resolution. What resolution is your data?
*Thread Reply:* We have annotations here as well. https://github.com/weecology/NeonTreeEvaluation If you would like to use them for your method.
*Thread Reply:* +1 for the Lidar point cloud argument, with a few returns per square meter it get complicated. to segment at scale
*Thread Reply:* @Jason Holmberg (Wild Me) - werent we just talking about this a couple days ago?
*Thread Reply:* https://www.mdpi.com/2072-4292/14/6/1317 here is an example of that LiDAR challenge. A extremely thorough well done paper, probably a year's worth of diligent work. Is only about 5% better than a completely off the shelf RCNN that you could run in a couple days. I would be willing to guess that if that's true, than the pretraining we've done on deepforest is probably equal to that, and this was in higher resolution LiDAR data. Its strange, and I do hope that eventually a massive, multi-sensor with RGB/CHM/HSI pretrained algorithm will outperform, but I think the labels are lacking to truly evaluate it.
*Thread Reply:* @Daniel Davila thankyou for this link, I have not seen it before. I am working with forest data, my search has been limited, haven't thought about urban tree data.
*Thread Reply:* @Ben Weinstein I have been pouring over this new paper last few days, the one you have linked here on urban individual tree detection from lidar point cloud.
I have not started working on this project too long ago, new to lidar point cloud. My actual data is ALS point cloud, and I want to segment individual trees. There hasn't been a ton of study on application of deep learning to segment individual trees in point cloud. I am going about it systematically, trying out the projection based method, ie. taking the bird's eye view, getting the 2D raster, applying image based instance segmentation. I had hoped that masks would better delineate a tree than bounding boxes. However, at this stage, I can appreciate your conclusion on masks not really improving on bounding box situation, specially when the trees are dense. I may have to abandon the Mask-RCNN idea, and look at 3D object detection/3D instance segmentation algorithms. Another incentive to first start with a Mask-RCNN was to establish a base-line, to compare with another approach.
*Thread Reply:* @Jon Van Oast if you and other people are working on point cloud of forests, would love to know more about your work! I am interested in "point cloud", "deep learning" and "segmentation"!
*Thread Reply:* @Kasirat - at this point we are just at the very beginning of wondering what work is out there. so this thread is great!
Hi everyone! You may have already seen this, but in case it didn’t make it on your radar - our DS4E Initiative at Berkeley is hiring an executive director!
I hope some of you may find this position interesting, and please feel free to share with colleagues. You can find more about the initiative here, but I’m happy to answer any questions you might have.
*Thread Reply:* Please, please be sure to post a link to the review here once it is published.
*Thread Reply:* Sounds like a relevant review, very complementary to our perspective paper! If you want have a chat on do’s and don’ts when writing a paper like this, just PM me!
*Thread Reply:* I second that; happy to talk about it!
*Thread Reply:* @Monty Ammar it's not a scientific article but sharing here a recent report on the use of AI for the NZ environment in case it's of any relevance for your review https://aiforum.org.nz/2022/05/18/ai-for-the-environment-in-aotearoa-new-zealand/
Hi all! - If anyone is looking for an accessible way to learn or practice with computer vision for conservation, a couple data scientists on our team just put out a practice wildlife image classification challenge with some camera trap images and 8 classes.
The data comes from partners at the Wild Chimpanzee Foundation and the Max Planck Institute for Evolutionary Anthropology. There is also an introduction to image classification using camera trap images blog post that walks through an initial approach to the challenge.
Please feel free to share with any students, data enthusiasts, etc. who might be interested in getting some practice in this field! Thanks!
@gvanhorn is giving a talk today on Merlin Bird ID!!
*Thread Reply:* Congrats Sara! Your defence was a great ride through all the amazing work you've done! And only the start of an incredible career ahead 😄
*Thread Reply:* Congratulations!!!! Can't wait to hear what's next.
*Thread Reply:* Such a good talk! Congratulations, Doctor!
*Thread Reply:* Congratulations Sara!! I had to leave at 10 but any chance you can share the recording?
Two post-doctoral opportunities for "AI in ecology" in Utrecht, Netherlands (PS I don't have any links with the institution or know more details about the positions, I just came across it) https://www.iamexpat.nl/career/jobs-netherlands/research-academic/two-postdoctoral-positions-ai-ecology-10-fte/391358
*Thread Reply:* Hey Amir - do you want us to pop this on WILDLABS as well, so the broader conservation tech community can see it?
*Thread Reply:* @Stephanie O'Donnell yes, please! 🙂
*Thread Reply:* Does it have a close date?
*Thread Reply:* I've put to close at 30June, but let me know if I need to change that! https://www.wildlabs.net/career-opportunity/post-doc-multi-sensor-fusion-animal-biomechanics
*Thread Reply:* thank you so much Stephanie! 30 June is perfect 🙂
*Thread Reply:* Hey! I think @Holger Klinck and @gvanhorn are bird enthousiasts, maybe they can help you out! Otherwise, try the #jobs channel. Then another question is probably to know what you are looking for (a master, a phd, an engineer job?)
*Thread Reply:* Hi Devis, thanks for the feedback!
https://crcs.seas.harvard.edu/get-involved
*Thread Reply:* If anyone here has experience with these I'd love to hear how they stack up to Jetson / Coral options for edge
*Thread Reply:* For ease of use, the Percept is pretty good. I haven't had a chance to compare performance across these platforms. @Henrik Cox (Sentinel) & @Sam Kelly, did you do any performance or efficiency comparisons across Jetson, Coral, etc.?
*Thread Reply:* We tried out the Jetson a while ago and have settled on the Coral since. Still using to this day
*Thread Reply:* This camera trap solution comparison doc by @Petar Gyurov is a great start on that Carly! https://www.notion.so/Camera-Trap-Pipeline-Solution-Comparison-2eac80825c4941b0b2b5fad3daea1cc3
*Thread Reply:* Also Dan's helpful running resource for overviews of solutions: https://agentmorris.github.io/camera-trap-ml-survey/
*Thread Reply:* Oh awesome hadn’t heard of these yet, thanks for sharing! Will def add these to the next update of the Conservation Tech Directory. I think the only ones I can think of that aren’t on Peter’s list are TrapTagger, FASTCAT cloud, ConservationAI, and obviously this new one Peter mentioned (Zendo).
*Thread Reply:* Oh these are great lists! Thanks for sharing.
The main difference between our platform, Zendo, and other options is that you can train it on your own camera imagery really quickly, and get a highly accurate model fit to your dataset and species of interest.
That model you've trained is hosted behind the scenes, and accessible for use at scale within the platform, or via an API.
You can then collaborate on and review new images within the platform, or export everything to a CSV file.
Looking for a postdoc opportunity using AI, sound and image recognition for nature conservation? We're about to advertise 2 x 2-year postdocs here in the beautiful Dutch city of Leiden at @Naturalis_Sci. Feel free to message me/email me if you want to know more
}
Twitter
*Thread Reply:* ha - great minds think alike, I posted it in the jobs channel today too😂
*Thread Reply:* Thanks Carly, I'm new here and didn't realize that there is a job channel. Thank you so much 😘
*Thread Reply:* Yes definitely check out the other channels! The ‘jobs’, ‘upcomingevents’ and ‘newpapers’ are particularly helpful for me in staying up to date with what’s going on in the field 🙂
*Thread Reply:* I wish! I always enjoyed visiting Naturalis during the Dutch part of my life.. say hi to Rutger Vos, if he is still there!
*Thread Reply:* @Thiago Bicudo, which job channel ??
*Thread Reply:* ah, thanks, it didn't pop in the channels list before!
*Thread Reply:* not all of them necessarily appear at first - when I first joined I had the same issue! I had to go to the “+” button next to channels --> ‘browse channels’ and then I saw a bunch of other ones I joined
*Thread Reply:* exactly… it’s a discovery process 😛
*Thread Reply:* Welcome! I’m in Delft too on a sabbatical!
*Thread Reply:* Hello Subhransu, I loved Delft!
*Thread Reply:* Assuming you know about NatureMetrics? If you wanted to do eDNA based aquatic monitoring they’re doing that already!
*Thread Reply:* Yes! Very aware of them since the beginning of NatureMetrics :) The niche market or the customer base I am thinking about is a little different than theirs.
*Thread Reply:* Hi Bilgenur! Welcome! A project Im working on involves using eDNA within a very large MPA to assess biodiversity, and working with NatureMetrics is problematic for a few reasons. We would welcome new players in this field, and in general our team would be happy to chat more! If this is of interest, please DM me and we can chat
*Thread Reply:* I’d also recommend posting to the WILDLABS eDNA group! Lots of people working on really cool stuff there. https://wildlabs.net/groups/edna-genomics
*Thread Reply:* Hi Ando! Thank you so much for your response. I would be happy to hear more about your project!
*Thread Reply:* Will do, thanks for bringing this group to my attention, Carly!
*Thread Reply:* Just came across this and figured it may of interest as well - https://www.ednacollab.org/ (they also have a grants program)
*Thread Reply:* I talk about the effect of climate change on the African rainforest. Especially the impact it has on forest elephants and why that is causing more Human-Elephant-Conflicts. Obviously not in much detail, because I only had 14 minutes to talk 🙂
I can tell you that this was by far one of the coolest but also scariest talks I've given so far. 😇
*Thread Reply:* Congrats @Thijs!! I can't wait to watch!
*Thread Reply:* congrats @Thijs! I gave one last year in Martigny and I agree 100%: super scary but also very rewarding
*Thread Reply:* Cool @Devis Tuia I'll try to look up your talk on YouTube 👍
*Thread Reply:* Ah it's in french, I don't understand that 😇
*Thread Reply:* Hey @Thijs! I’d love to see your talk. I looked for it online but couldn’t find it, do you know if it’s been posted? Thanks!
*Thread Reply:* @Colin Donihue The talk is not online yet, this usually takes 4-8 weeks for editing and reviewing by TED. Once it's online I'll post it here 👍
In case you are around at CVPR2022, please let me know! It would be super cool to have a AIforConservation meetup (for example for lunch once)! you can find me for sure tomorrow (Sunday) at the Earthvision workshop: https://www.grss-ieee.org/events/earthvision-2022/
*Thread Reply:* Hi @Devis Tuia i am at CVPR, too. As probably a few others from this slack channel.
A few are probably attending the CV4Animals workshop tomorrow. How about we get something for dinner afterwards with anyone who is interested?
*Thread Reply:* Cool! Let’s try lunch first at the cv4animals break? Like we can meet at the end of the session just outside of the cv4animals room? 11h45?
*Thread Reply:* My colleague, Matt Dawkins, will be giving a talk at the CV4Animals workshop in a little while here. We have a lot of team members from Kitware milling about the conf and are at booth #1522 in the exhibition, if anyone wants to meet up! Have a good week yall.
*Thread Reply:* @Frederic let’s discuss at the end of the sessions of cv4animals. I am in the room btw 😉
*Thread Reply:* sorry did not see your message. Lets meet afterwards, to discuss it.
New paper on transfer learning with passive acoustic data out, led by@Emmanuel Dufourq! Happy to have played a small co-author part but he deserves the recognition for all the programming heavy lifting! https://www.sciencedirect.com/science/article/pii/S1574954122001388
Auto Arborist has been released!! This fine-grained multiview dataset contains over 2 million trees belonging to over 300 genus-level categories in 23 cities across the US and Canada built to foster the development of robust methods for large-scale urban forest monitoring
https://twitter.com/sarameghanbeery/status/1539703332100521984
Twitter
}
Twitter
Folks, does anyone have connections to individuals in the following orgs. Please let me know if you do, so I can DM you for a small favour.
350.org AfD APANO Citizens' Climate Lobby Climate Action Rhode Island Climate Justice Alliance Climate Pop Up Creative Action Network EARTHJUSTICE EDF Environmental Defence Canada Environmental Defense Fund ENVIRONMENTAL WORKING GROUP Extinction Rebellion Delaware Extinction Rebellion Malvern Extinction Rebellion NL Extinction Rebellion Québec Extinction Rebellion Skåne Extinction Rebellion UK Fridays for Future Fridays For Future Toronto Friends of the Earth Friends of the Earth Europe Friends of the Earth Malta Friends of the Earth Scotland Greenpeace UK Greenpeace USA HEALTHY ENVIRONMENT ALLIANCE OF UTAH Israel Action Network LPAC Action Network Michigan Climate Action Network MTÜ Loodusvõlu Natural Resources Defense Council NRDC RAINFOREST ACTION NETWORK Rana Sunrise the Environmental Working Group Toronto Climate Action Network Union of Concerned Scientists USA Justice Alliance World Wildlife Fund
*Thread Reply:* Specific departments or areas? Some of those are very large & multi-faceted so I would have a better idea if I know someone with a bit more context/detail
*Thread Reply:* Yeah, I do know some people at some of these organisations. But it's quite a long list. Maybe you can elaborate a bit on your intentions?
*Thread Reply:* Thanks @Carly Batist and @Thijs. I am looking for someone dealing with Facebook advertisements at these orgs.
*Thread Reply:* so someone in the marketing or comms departments then?
*Thread Reply:* And lets assume you find people at these departments, what's your question?
*Thread Reply:* I am interested in talking to them about how they advertise on Facebook. 🙂
*Thread Reply:* Do you work for an org that is trying to do this? Just curious how you came up with this list
*Thread Reply:* I am a PhD student at MIT, looking at how environmental orgs communicate about CC.
*Thread Reply:* This list is from my initial survey of adv orgs. Happy to talk to other adv orgs who advertise/post on FB too!
*Thread Reply:* disregard my emails highlighting…(just noticed that🤦♀️
🗣️🔊🎶 For those interested in AI for bioacoustics/ecoacoustics applications 🗣️ 🔊🎶
The Bioacoustics Stack Exchange has reached the Beta stage and is LIVE! It is also in a critical stage, and we need your help to keep it alive. We have only 2 weeks to prove we have the community to support the site so please help as you can! If you want more information on how to help or get started with Stack Exchange, check out this helpful Youtube tutorial made by Selene Fregosi.
There have been LOTS of questions on AI/ML for bioacoustics recently so would be great to have input/users from the tech side contributing as well!
How can you help? • JOIN: https://bioacoustics.stackexchange.com/ • VISIT: Make it a habit to check in on the site regularly. We need >>500 visits per day! • VOTE: We need to show participation! Vote on all good Q/A • ASK: Have a new bioacoustics-related question? What about your older questions? You can even ask/answer your own question! These questions will be archived for future use, so ALL good questions are welcome! We need >> 10 questions per day! • ANSWER: Have an answer to one of the questions, or additional information? Share your expertise where you can! Is there a question you are often asked? You can pair up with someone to ask/answer it again to be archived! If someone asks you a question-- tell them you will answer it on the Stack Exchange! We need ~2.5 answers per question. • INVITE & SHARE: Invite others to join! Share on social media, with your collaborators, etc.!
Hello, a TREE paper on new techs and insect monitoring with an important focus on AI, computer vision and acoustics: https://www.sciencedirect.com/science/article/pii/S0169534722001343
NSF funding opportunity: https://beta.nsf.gov/funding/opportunities/partnership-advance-conservation-science-and-practice-pacsp.
Hey everyone - recently posted my take on the role of biodiversity monitoring tech in scaling up private sector investment into nature... would be super interested in expert feedback! thanks 🙏 https://www.linkedin.com/posts/georgedarrahwe-need-to-bring-biodiversity-to-the-bloomberg-activity-6950070143146209280-e59v?utmsource=linkedinshare&utmmedium=memberdesktopweb|https://www.linkedin.com/posts/georgedarrahwe-need-to-bring-biodiversity-to-the-bloo[…]e59v?utmsource=linkedinshare&utmmedium=memberdesktopweb
*Thread Reply:* Agreed - but my sense is that without private sector funding and significant change to taxation we will not turn the tide on biodiversity losses
*Thread Reply:* So lobbying both industry and government is needed, and keep raising awareness in the public and even in the academic spheres.
Hey everyone! Bit of personal news from your friendly neighborhood AI for Conservation founder - I've accepted a faculty position at MIT, starting fall 2023! I'll spend the next year at Google working on multimodal/multiview urban forest monitoring (see Auto Arborist), but I'll be hiring PhD students and a PostDoc this fall to start with me next year! If you're looking for an academic position at the intersection of CV/ML, biodiversity, and the environment please reach out!! I'm looking to hire students from diverse backgrounds to work collaboratively on interdisciplinary research projects.
Twitter
*Thread Reply:* feel free to join the #marine channel all about CV particularly for marine things!
*Thread Reply:* Where are you based geographically now? That would help with identifying projects
*Thread Reply:* We are both located in Texas, and we are currently seeking any remote opportunities!
*Thread Reply:* Try your best to not settle for unpaid internships. You deserve money for your efforts!
*Thread Reply:* We will be happy to just gain exposure and learn at this point of time, but maybe in the future! 🙂
*Thread Reply:* If you live in a town with a university, see if there are any labs or professors working in the AI for good space and reach out. Many professors I know have high school students working in their labs or work with high school programs so it’s worth a shot to ask! You might also check out the Conservation Tech Directory to scope out possible organizations/companies/projects to reach out to.
*Thread Reply:* I have to echo Mark, it's actually more and more common for companies to take on high school talent. If you are contributing value to the enterprise, you deserve compensation!
That being said, what part of texas? There are a ton of great universities out there, the UT system for example, with a campus in every sector. There is a very large research institute in san antonio, SWRI, that may have programs for your level. You could also just jump into an open source citizen science project if you have your heart set on free labor, there are so many out there.
*Thread Reply:* Thank you all very much for the guidance; I greatly appreciate it. I will also definitely consider looking around for paid opportunities as well!! 🙂
*Thread Reply:* Also, huge congrats on being done with the job market and landing a spot at A&M!
*Thread Reply:* I’m exited to join this group 😄 Anyway, welcome 🙂
*Thread Reply:* PS … my image was the result of asking DALLE-2 to produce “a.i. for wildlife conservation digital art”
*Thread Reply:* @Ben Weinstein I don't have a good answer for you but your own application is identical to one of ours, which is identifying seal species classified from a computer vision model to get a good estimate of species-specific abundance. There are two layers of uncertainty: 1) the classification of an seal (i.e. is there anything here at all?), and 2) its classification to species (if yes, what is it?). We've had a working group here at IACS of people interested in this exact question at the interface of ML and Bayesian inference because we are now combining ML classification with population models. Unfortunately, the two disciplines are quite far apart and a common understanding of error propagation methods is difficult to come by.
*Thread Reply:* Don't forget that species classifications are often disputed and have substantial consequences for conservation planning/decisions! Value choices and uncertainty enter into the classification model from multiple inputs sources
*Thread Reply:* @Ben Weinstein When thinking about “When are these intervals useful?”, could you approach this as a model selection problem? Specifically, could you allow an end-user to make various model assumptions to predict occurrence, then demonstrate performance relative to alternative models within a tool framework? Happy to discuss more.
*Thread Reply:* For my research I am looking at something similar, large terrestrial mammals in savanna biome. The first portion will be focused on establishing accuracy and precision from known populations. Once this has been determined it should be possible to scale up (assuming similar characteristics) and gather reliable data on population sizes and distributions. For vegetation I assume you're using satellite imagery and EVI/NDVI for some kind of semi-supervised classification?
*Thread Reply:* I wanted to add this paper here because it was thoughtfully done.
*Thread Reply:* Hi Jitendra, I should be able to help you with that, I am working on a Data curation platform; sending you an email with the information
*Thread Reply:* Hi Jitendra, this sounds fantastic! Also just sent an email to see if there's a good fit with the work we're doing
*Thread Reply:* Hi Jitendra, I believe we at WildEye could help you out with TrapTagger - our free-to-use and open source platform. I have sent you an email with more details.
*Thread Reply:* In terms of the loss function, you can try to use focal loss for single shot object detectors. For 2 stage, you don't need to handle that. In terms of data processing, bounding boxes are usually regressed as anchor box offsets, and anchor box locations are fixed. Which is independent of the class label. One thing you can try, is use an indicator variable and don't penalize the loss on the bounding box prediction if the pixel is labeled as background.
*Thread Reply:* Hi! What typical object detectors do is to simply ignore all the bounding boxes that are predicted for background. The way this is done depends on the model type: • Two-stage detectors (e.g., Faster R-CNN) check the intersection-over-union of each detected bbox with the ground truth; if it falls below a threshold it gets ignored. • One-stage models (esp. YOLO) predict bboxes in a grid over the image; here, the bbox that is predicted in the same grid cell as the ground truth (and has best overlap) is used for training; the rest gets ignored. What is always trained is the class score (and objectness, if available), because these are defined for every bbox (same principle as above).
By the way, 0 is not automatically reserved for background. This depends on the implementation—for example, Detectron2 uses 255 for the background. Also, some models have no background class at all: RetinaNet has sigmoid-activated class outputs and thus does not need to predict an extra background class.
*Thread Reply:* You can also look at some of the newer anchorless approaches (e.g. FCOS, Centernet2, etc...) which densely classify each pixel, and then apply the bbox loss to just the positive pixels.
*Thread Reply:* thanks a lot everyone. makes much more sense now. i was following the torchvision FRCNN tutorial to implement this and they mention to set the background class=0 (as reserved and not to be used for other classes) but don't say anything more on it.
so basically if the RPN network in a 2 stage model has no "objectness" boxes passing the criteria then the class-box prediction network has nothing to predict and would return nulls
has anyone used https://labelstud.io/? I'm starting to looking into open source tools (or with cheap 'enterprise' add ons) where we can start creating active learning and model verification environments. We have been using zooniverse (both public and private) or locally annotating in QGIS for a few years and I think we are maturing beyond those tools where we want direct model integration and some control. Or atleast a toy model we use to guide annotations, which we than download and develop more novel models offline (more likely). I could build something from scratch (or piggyback on awesome tools like AIDE @Benjamin Kellenberger). Just a hypothetical for the moment.
*Thread Reply:* We've been working on something like this with @Tom Bernardin and @Caleb Robinson for remote sensed building detection
*Thread Reply:* sounds good. I think my general question is whether we should be building something or trying to rally behind some reasonable open source starting point.
*Thread Reply:* Hi Ben, I could connect you with my team if you want a demo of Caleb's tool. It is open source, as are (or will be) the modifications we made for our use case.
*Thread Reply:* Ben, we can definitely talk for AIDE, to see if it can align to your needs. If you want we can setup a meeting together with @Benjamin Kellenberger.
*Thread Reply:* I am a big fan of CVAT and can help you if you would need some support or insights. They have a lot of integrations for automatic labeling functions using different frameworks or completly custom code. Good api, … https://github.com/openvinotoolkit/cvat
*Thread Reply:* i've just started using label-studio and really like it.. for the specific question, i would tend to lean towards using existing opensource tools.. simply because, like in case of label-studio, it does all the heavy-lifting allowing you to spend time and resources on your main goal rather than re-inventing the wheel.. it was even so easy to deploy it on the cloud and serve it to annotators remotely..
*Thread Reply:* I keep checking to see if label studio has added a spectrogram option to their audio annotation features. There has been a ticket on their github for quite awhile.
*Thread Reply:* I've been using the free version of Label Studio for both text annotation and image pair annotation, albeit without any model integration to speed up/guide labeling. Overall it's not a bad starting point, but I do think a lot of useful functionality is only available in the enterprise version, which is a little frustrating.
*Thread Reply:* @Amrita Gupta do you get the sense that the model integration is possible without the enterprise version? It was unclear from the docs. In general i'm leaning towards label studio, but I just got word today that a potential next project has been using CVAT and i'll probably be asked not to re-invent too much, @Frederic I can imagine three avenues for model integration, 1) deciding which images are shown next, 2) pre-label images with existing markers, either from a live model, or a batch upload, 3) decide what portion of the image is to be shown, perhaps integrating with cloud geotifs. These are going to be large airborne tiles with mostly background. Does that sound about right?
*Thread Reply:* Yes model integration is possible with the free community edition (but you have to wrap your model code into a LabelStudio ML backend server class). You can then train/fine-tune models and get preannotations, which is enough to "hack" batch AL functionality. Online AL with continuously training models seems to be only available through the Enterprise version.
*Thread Reply:* Thanks @Devis Tuia, @Benjamin Kellenberger had talked about this sometime in past. I see three avenues for active learning (copied from above) 1) deciding which images are shown next, 2) pre-label images with existing markers, either from a live model, or a batch upload, 3) decide what portion of the image is to be shown, perhaps integrating with cloud geotifs. I wasn't sure whether we should go the truly open source route (AIDE, CVAT) or try to hack into a more commercial application. I haven't even gotten the final funding for this yet, so i'm just in the information phase.
*Thread Reply:* @Ben Weinstein Regarding your questions, cvat was not designed for this active learning approach, anyhow using its api or db access you could integrate something that might work. Anyhow the speed might not be great.
1) deciding which images are shown next While having a loop fetching labels from the api, you could add images to a job based to the finished annotations. Possible issues: the browser of the client might have a cashed version of the task and need to retrieve the changes.
2) pre-label images with existing markers, either from a live model, or a batch upload, Pretty easy and you can use different approaches:
- use the api to create tasks, upload images and annotations.
- If its a commen model and task you can just use the serverless functions and replace the example models with your models. 2.1 Since https://nuclio.io/ is used for serverless function, you can build your own nuclio container for labeling.
3) decide what portion of the image is to be shown, perhaps integrating with cloud geotifs.
- use the api to upload cropped geotifs
- You might be able to set Zoom block via the api. but not on a individual image basis. 😕 -> GeoTIFS and image tiling is not supported right now see, but you can implemented and get help from the community https://github.com/openvinotoolkit/cvat/issues/531
Overall CVAT might not be perfect but allows you to hack it, to make it work 🙂
*Thread Reply:* Bit late to the party, but for what it's worth...
I've used both CVAT and Label Studio, and we've been using a commercial platform at work. We're also looking into active/unsupervised learning. In the past I've written labelling tools out of frustration with some of the limitations of open source stuff. My gut feeling is that it's sensible to split out functionality to the tools that do it best (e.g. storage, labelling, ML annotation+inference and active learning/sampling). The risk is that a lot of tools are almost there, but they cater to common use-cases that don't have a huge amount of complexity and none of them do everything perfectly.
Label studio is pretty good. I've used it in a pinch to do some semantic segmentation, though there were some funny GUI issues if you had overlapping polygons. I think for most part all of these tools are pretty good at tagging and bounding box annotation. The differentiators from a labeller perspective are around how quickly you can segment objects. CVAT is also good and has a usable, if tersely documented, REST API. Error handling is... cautious? For example if you upload images into a project and then upload labels, you must have labels for all images (even if empty) or the importer breaks.
In terms of integration and on the "ops" side. I think there is a lot of smoke and mirrors from commercial platforms about exactly what is implemented and how well - especially when they're charging you $100+/seat/month. A lot of companies will try and sell you black-box ML integration which can work very well within their walled gardens, but have issues like:
• Can we adjust anything about the model? Often no. • What model you use? Proprietary, but probably Detectron2 • Does it work with some weird TIFF that we have? Maybe • Does the service offer image tiling for large inputs, as is common in remote sensing? Often not. • Can we take out the model for use elsewhere/can we self host? No, or pay us a lot of money. • Can we specify exactly which splits are used during training? Often no • The list goes on. Most of these are non-negotiable unless you have a huge contract and pull on where development goes. At least with open source you can see how the sausage gets made. The downside with larger projects like Label Studio and CVAT is that even if you have the technical capacity to PR things, it can take a while to get integrated.
There are other toolkits like Lightly who focus more on the active learning side of things. A lot of their unsupervised learning code is open source. Some functionality may be available for free if you're a non-profit. They're based up the road from me in Zurich and I've met them - they're a nice competent team, growing fast. Also fiftyone which is supposed to be good for model assessment, but really struggled when I imported a moderately sized dataset into it so I can't say much there.
I think Frederic's last answer is sane. You can get away with a labelling tool that has a relatively complete API for file management (like CVAT) and offloading the sampling and annotation stages to your own code. Uploading annotations from a pre-trained model, before handing images to annotators, is what we do at the moment.
(3) is the difficult one to do well IMO. There are some companies who are selling GIS oriented labelling solutions (e.g. Picterra), but not cheap. I agree the simplest approach is to do that work offline and just crop/upload tiles. Though in principle there isn't any technical reason why you couldn't use something like rasterio to specify a window/windows into the image? At least some of this is offloaded in CVAT by datamuro, so you could look at adding an importer there that could take e.g. a single source image and a list of windows, and then have datamuro handle the tiling and forwarding to CVAT.
*Thread Reply:* I think this is a really excellent summary. We want control for own models for the active learning side. I will give label studio a try because it feels just insane to have to develop own tools for such a common process. If it looks like we just can't get the control we need, its either CVAT or AIDE. I asked a similar question last week to @gvanhorn who was giving a talk on Merlin and it is clear that Cornell have spent quite a bit of time developing their own tools from an existing open source repo.
*Thread Reply:* At some point I think it'd be worth the remote sensing community pulling together on this, because it's clear there's common functionality that lots of groups need that isn't really provided by the standard open source tools. Maybe we (as a field) need to sit down and figure out what the requirements are and make some recommendations, and then decide on how it gets implemented - or at least document some best practices. There's a huge group of people that would benefit - basically the entire EO sector for a start. My experience talking to labelling companies is that remotely sensed imagery is a bit of an unknown to them - unfamiliar formats, processing paradigms, etc. Though many have said that they have customers who are requesting more geo-oriented features and I think we'll start to see more support in the future. Same goes for bioacoustics I guess, my limited understanding is there aren't great open source audio annotation tools at the moment (unless you want to just tag/cluster spectrograms).
(edit: addendum to this is that virtually all the GIS tools and geo libraries that would need to do the complicated things - stuff like GDAL - are open source, like how every AV/media company in the world depends on ffmpeg... So in that sense I think a lot of the hard work has already been done and this is mostly a data serving problem?
One major pro for commercial tools is that the interfaces are often really well polished because they have a team of frontend devs working to make things look good, and the labelling "experience" for lack of a better word is pretty good on some of these platforms. My worry about freemium tools like label studio are that you end up with features locked behind subscriptions and there's little incentive to make those public.)
*Thread Reply:* I wanted to restart this thread to ask whether anyone had any advice on how to serve megadetector's results through something like label studio or fyftyone to multiple annotators for manual correction and further annotation of the results. It seems that one obstacle is to convert from the COCO CT format to the classic COCO format?
Hi folks! Dan Morris just added me and its great to be here! Wanted to quickly introduce myself - I've been working with Felidae Conservation Fund, a nonprofit out in SF-Bay Area for about a year. Last fall, I migrated their data pipeline to the cloud and built a human-in-the-loop annotation platform using Megadetector & Annotorious (over Django, Dropbox & GCP). If you're curious about the approach, I've written a post about it here.
I'm looking for feedback & opportunities for collaboration! Please let me know if there is any piece that I can abstract out and open-source! We do have plans to open source some of the data after it gets cleaned up as well!
(Some broader context - My background is mainly in computing and AI/ML/DL but applied to NLP & RecSys and I also do full-stack web development. While I used to do more hands-on ML research, I've zoomed out to ML system design and in this domain, my focus is mostly on building usable plug-and-play open-source software layers on top of existing models)
*Thread Reply:* Welcome!
To your point about public data release, when FCF has data that is ready to share publicly (minus location information, and minus images of people), we would be happy to help get that up on lila.science (I'll even volunteer to do whatever data munging is required to get it into the same format we use for other camera trap data sets on LILA).
*Thread Reply:* Thanks Dan! 🙂 We'll keep you in the loop through that journey! Luckily, WildePod was set up with continuous model training in mind so it should be easy to export it out into COCO or other standardized formats for LILA!
*Thread Reply:* WildePod is very cool!
*Thread Reply:* Great post, @Abhay! I am on a similar journey myself with the BearID Project. While BearID is primarily focused on camera trap data, I am currently working on a full-stack web app to annotate and identify wild bears on the Explore.org web cams in Katmai National Park. Even though it would be better to adapt something, I have succumbed to the urge to build something from scratch, mainly to build my web skills and take advantage of being an AWS Community Builder. I have been documenting my journey on this blog. I was considering the
*Thread Reply:* Based on your screenshots, I need to up my UX game! 😄
*Thread Reply:* @Ed Miller and @Abhay have a look at the thread I started yesterday on annotation (https://aiforconservation.slack.com/archives/CLWGQ4BJ6/p1660171984124629). It feels like we are in need of some community development here. I sense that as a community we are moving from "does computer vision work for ecology" which was the question a few years ago (and motivated this work https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/1365-2656.12780) to "how do we develop iterative pipelines that tangible change long-term monitoring programs". I was giving a talk to @Sara Beery summer school on this kind of shifts in the community this week. I'm going to both read of your materials and try to write some summaries for the entire group next week. There are alot of competing goals and platforms here. @Jon Van Oast @Jason Holmberg (Wild Me) and wildbook have something to contribute here as well. Might be the inspiration for a small paper. I'm writing a grant on active learning this week and welcome input on how/should/if we should make something that aims for a more universal approach (is this even a maintainable/worthwhile goal) or ecological workflows are just too varied and sharing knowledge and research aims into active learning is a better strategy than tool building. i'm 100% open to thoughts, @Benjamin Kellenberger and I had a brief conversation surrounding this earlier today.
}
*Thread Reply:* Thank you! Also, if y'all want access to the system to get a feel for it, please DM me your email ids and I can add you in!
@Ed Miller - That is pretty neat! Happy to share my Annotorious code if you want. It is extremely simple and I'd highly recommend it especially when the number of options aren't many. You can even customize the UI a fair bit. In the gif that you see, it's simply using Bootstrap5 classes. And thanks, my UI is Bootstrap5 so the entire scss is maybe 100 lines mostly for colors and fonts 🙂
@Ben Weinstein - That was a pretty interesting discussion because they were essentially the same questions I went through when I was building this! I've gone about it in depth in the post but I'll add a TL;DR with some more thoughts that is very much in line with the discussion in the thread. So the broad context here is that most of the design & dev happened last Fall. I'm very strongly against building anything so at the time, Wildlife Insights was an option I seriously considered (other one was Zamba) but decided against it for two main reasons.
- The big one was that the nonprofit needed a more generic org-management system for inventory, the camera traps out in the field, its status of use, when it needed to be checked, potentially emailing volunteers to go check it etc. For this, I'd have to build a web app anyway and Django came with all this out of the box so it was not really any work.
- Next was access to models which is something you've touched on in the thread. I wanted an online-learning human-in-the-loop system. Since Megadetector was already mature, we wanted to use it and not even tune it since it was very likely they'd put out upgrades (as they did a month ago). So our goal was species detection where it seemed like the models were still not very mature. So the intent here was to create a basic model and then, as annotators did their job, have the model update at a regular cadence. Since other volunteers fell through and I got busy, we didn't get to it but do intend to do it this fall if there isn't already an existing solution. I'm not sure when Wild Me came about but I did learn about it early this year but WildePod was already up. The core goal for me was data standardization, ideally with object-level annotations from their existing images & camerabase annotations. After that, the hope was (and is) to keep useful pieces of WildePod and migrate elsewhere!
I strongly agree with the opinions in that thread about tool fragmentation. When I was first reading through Dan's blog post, I was surprised at how many custom solutions existed but it made sense given how nascent this is. Usually the tech matures first before the products and from my perspective, it looks like the models have matured with Megadetector/CVPR workshops but the software layer around it has some ways to go.
When I built WildePod, my goal was a "Bring your own storage & compute" approach to keep the expense around the website low. So the app is simply a layer around Dropbox. This way, any open source solution won't have to bear storage costs and people can link their own dropbox accounts. Another thing here is compute for inference. The Django app simply hits an endpoint so technically you can host your own model. Only downside here is that you might have to deploy it to the cloud yourself. I don't have an answer to the latter since it is a trade-off between cost of compute vs labor-cost for end user in terms of deployment.
Stepping outside the build scenario, since this is also a common CV task, there is tooling aimed at enterprises that have matured (labelbox, labelstudio, roboflow, scaleai, google's vertex AI etc.). Here I'd skew towards something like that when the dataset & the org is larger and has capital or if the company sponsors research/nonprofits. I had a conversation with Joseph Nelson, who runs Roboflow (invited him to this group) who reached out and wanted to see how they can help. They've had success with it in the past as well. In an ideal world, if there are small set of paid & free options that folks rally around, it'll greatly help standardize the data underneath. I think the key is data standardization so researchers can seamlessly jump vendors.
I'd love to chat and learn more about this. I've mostly done this in isolation while consuming content from the web (like wildlabs summer seminar series which was great!) so I probably have a lot of blind spots. I'm curious to know if folks here sync occasionally over zoom? If not, would there be any interest in doing so at some cadence?
*Thread Reply:* @Ben Weinstein the thread and discussion are definitely of interest for BearID. I definitely think the time is right for convergence, at least on no-cost/low-cost labelling. For the Bearcam Companion application I am working on, a big part of it is for me to become more experienced with some of the technologies and tools for web development. I am very much building much of it from scratch. Longer term, and for the BearID Project more generally, the more standard tools we can utilize the better.
@Abhay I am interested in your code. I am especially interested in how you are saving data for different users (labelers). Can different users label the same image? In my case, for identifying the individual bear, I am allowing each user to select a label when editing, then provided a tabulation of "votes" in the main view, e. g. "480 Otis (4/6 = 66%)." I guess I could leave my current method for the top view and only use Annotorious for the editing. I will need to connect this to AWS DynamoDB through the Amplify DataStore API. It looks like I can use the Firebase as a reference.
*Thread Reply:* Sorry just saw this.!
Yup. Multiple users can label the same image. That piece of logic is fairly primitive at the moment. As of now, the system will throw images with the least number of votes at users. The users can then add/update/delete boxes. This actually goes into the system as a "vote" per box.
So when another user sees the same image, they will see a union of boxes created by machines and humans.
We have a hardcoded threshold for now (~2) as the vote difference. When that threshold is breached, boxes will not be shown anymore.
Right now the weights are uniform. As things mature, we'll very likely add some weights to annotators depending on tenure/success rate etc. to soften the hardcoded threshold.
*Thread Reply:* For the data part, since everything is in Django, it is simply writing it to and from the Django models. Underneath this, the database is actually Postgres hosted on CloudSQL. All this is fully abstracted through Django's ORM.
Depending on the infrastructure you're using you might find language specific ORMs to make write-read easier.
The biggest overhead here is usually CRUD on these tables. If the service makes it easy, then great. If you are maintaining a single row of data along the lines of <image> <user> <annotation> with mostly appends & no related models, I'd recommend skipping relational DBs altogether and write directly to a beefy spreadsheet like Airtable (it has a simple python API)
(In my case, I had to manage user accounts, inventory, camera traps etc etc which required a basic relational DB)
*Thread Reply:* @Abhay I integrated Annotorious to create/edit/delete the bounding boxes on my images. My website is build on AWS Amplify and React. It took me less than an hour to integrate with my UI and data store. I was able to add a label vocabulary, but the user has to start typing. How did you set yours up to only have the selection options you want?
For now, editing objects is an "admin" feature. I'm not dealing with multiple users yet. I'll need to expand my Objects tables to enable per users labeling. I already support that for the bear identification though.
*Thread Reply:* That looks great! Since MegaDetector only has three classes, I render them as buttons. If it is a long list, you're likely be left with the option list. You could even try a mix of the two by rendering the top-K options as buttons with the option to choose more from. You can do all this by customizing the widget [instructions here]. The example has the template to add any arbitrary html element you want (and also style them the way you want). (I also DMed you my widget to see what modification might entail)
*Thread Reply:* Hi @Abhay! I was wondering whether the source code for this project is available somewhere? Your app demonstrates how to serve MD results to multiple users and I haven't seen any other examples like that.
*Thread Reply:* Hi Valentin. Just saw this. I have two repos up
- https://github.com/hayabhay/megadetector-fastapi - This is a fastapi wrapper with a dockerfile that is ready-to-deploy on Google Cloud Run. It can essentially run any MD models (you can choose which one to bundle in docker) and can arbitrarily scale to users with GCR. (this is already running for us). Only downside is that GCR doesn't support GPUs so you'll be doing inference on CPUs (~15ish seconds per image). It also comes with a Streamlit UI to quickly test things
- I also have an older version for MD v4 that was deployed as a Google Cloud Function. Again, its very similar to GCR but it doesn't require an api wrapper. Feel free to let me know if that works for you and if you have any custom requirements. I'll be more than happy to work with you to get MD up on the cloud (my knowledge is limited to Google Cloud Platform however)
*Thread Reply:* Also, in case I misread your question and If you're asking about the Django app that does human-in-the-loop annotations, it is right now in a private repo since it has some coupling with Felidae's branding. If you're interested, I can put out a ready-to-use open source version for it as well or if you're just interested to see the code and take pieces of it, I can add you to the org's Github. Let me know what works best. If you'd prefer to have a quick chat, let me know and I'll be happy to get on a zoom call with you! (also adding this to the wider channel in case there are more use cases)
*Thread Reply:* Hi @Abhay! Thanks for the links, I've seen them a while back and already starred them 🙂 I am indeed asking about the django app side of things. I understand that a certain IP is attached with your app and that you may not be able to share it all. One of the options we are considering for our project is to run MD on a research cluster (on GPUs) for speed once, and then serve the results (the images with the bounding boxes) for review by multiple users in our research team. That would require building an app similar to yours that would just serve the images with the boxes, offer the user the ability to vote for a species ID (I like your voting system) and possibly other tags (sex, age class) and also edit/add bounding boxes if any are missing (hence my interest in your integration with annotorious). I don't know if that is clear, but basically my webdev skills are limited and I'm exploring the feasibility of that idea on my end. Seeing the ORM model structure of the django app would be very helpful for me for example.
*Thread Reply:* So your offer of sharing ready to use version or to add me to the repo, as well as chatting, all or any of that would be of great help for me 🙂
*Thread Reply:* Oops! Sorry for the initial misread! And yes, that sounds great. We can do any or all of it depending on how you want to proceed. The larger codebase is a bit gnarly since it does have a fair number of ad-hoc product related patches. Regardless, I can walk you through the code as needed and/or extract specific bits of it to fit any existing apps you have.
To start, I can add you to the repo so you get a sense for it. Then, we can quickly jump on a call so I can get a better sense of what you're planning to do and from there we can figure out how best to move forward. Also, we have a new volunteer who is working on the Django side of things as well and there might even be room to build reusable components along the way.
*Thread Reply:* Awesome, I will send you a DM!
*Thread Reply:* hi paul! what taxa are you working on.
*Thread Reply:* Hi Paul, welcome to the community!
*Thread Reply:* Thanks! I'm looking at large (>50kg) mammals
*Thread Reply:* Out of curiosity, what happens when you apply our 'bird' detector to such images. https://deepforest.readthedocs.io/en/latest/bird_detector.html We'd like to formalize this towards an 'animal' detector in the next year or so.
*Thread Reply:* Hello @Paul Allin, I'm Fadel from Switzerland (EPFL) currently working on my Master thesis on "automated aerial census of large herbivores". I would love to connect and discussion this topic with you and learn more about your work.
*Thread Reply:* Hi Fadel, great to hear about your masters. Maybe easiest to set up a call. Can you WhatsApp me? +27712287116
*Thread Reply:* Hello @Fadel good to hear from an EPFL colleague 😄. In which lab are you?
*Thread Reply:* Hi Luke. I work on an open-source camera-trap-processing web application called TrapTagger . At present, we only offer an in-house southern-African-species classifier, with MegaDetector as an empty-image remover for our users outside of Africa. However, we are looking to add more species classifiers and are in the process of collaborating with a few organisations to host their classifiers for both their own use, and the use of others. Perhaps we can assist you by providing you with a GUI and annotation workflow to wrap around the species classifiers you are working on, or you could even just provide your classifiers for others to use free of charge if you like. Send me an email at nic@innoventix.co.za if you would like to set up a discussion around this.
*Thread Reply:* Hi Luke, that sounds like an awesome project. I am based in Oregon but working remotely with several automated insect monitoring projects based in Montreal and the UK. I am handling the software and infrastructure side of things, but also hoping to meet some folks and setup some edge devices with partners in the Pacific Northwest.
*Thread Reply:* I'll take a look at TrapTagger as well @Nicholas Osner! I am working on a desktop application with some similar functionality, but focused on moth and insect classification.
*Thread Reply:* Hi Michael, I'm sorry - I managed to miss your message until now. Let me know if you would like to have a chat about TrapTagger. Maybe you can make use of some our open source code. Worst case scenario, we have a bit of a chat about our respective projects. Let me know.
*Thread Reply:* The biggest gotcha while migrating was using YOLOV5.
Since imports within the repo isn't relative, the repo root requires being in PATH/PYTHONPATH .
As a result, yolov5 packages aren't namespaced and having a top level module like utils will lead to non-obvious errors. So any referencing code must have it's own non-conflicting namespace at the root level.
*Thread Reply:* Also, there was a fair bit of setup in terms of cloning the repo, checking out a particular commit and setting up the env. I've replaced it by automatically downloading the archive zip file and setting the necessary env from within the MegaDetector code itself.
*Thread Reply:* FWIW all of that stuff with checking out a particular commit and a particular version of PyTorch was working around this issue:
https://github.com/ultralytics/yolov5/issues/6948
...an incompatibility between certain versions of YOLOv5 and certain versions of PyTorch. It appears (in bold because I'm not 100% sure yet) that this has been resolved, i.e. that new versions of PyTorch and new versions of YOLOv5 now get along nicely and can run MDv5. YMMV. We'll update instructions if we're sure about that at some point.
*Thread Reply:* Ah! Interesting! I did forget to mention that the version of PyTorch was also a gotcha and that was more to do with PyTorch's compatibility with yolov5 (even with the specific commit).
I had to use these versions for it to work
torch==1.9.1
torchvision==0.10.1
*Thread Reply:* Also see this discussion:
https://github.com/ultralytics/yolov5/discussions/9138
I verified this in one (non-GPU) environment, and everything went fine. Need to verify in GPU environments.
*Thread Reply:* Ah! I think the latest torch version that I used (torch==1.12.1 ) triggered this error AttributeError: 'Upsample' object has no attribute 'recompute_scale_factor' (for the specific commit as well). Looks like it is still an open issue.
Also, I see the No module named 'utils' was mentioned here (likely from not being in the path). For me, this was because i had a utils.py in my own API that prevented it from being imported (since I had yolov5 in sys.path before the my project root).
<a href="https://github.com/WongKinYiu/yolov7">WongKinYiu/yolov7</a>
*Thread Reply:* check out LILA BC - they have tons of animal camera trap, audio, etc. datasets that are labeled and ML-ready. https://lila.science/
*Thread Reply:* Explicitly labeled distractors via bounding boxes or segmentation masks isn't something that exists already for any of the LILA datasets as far as I know, but camera trap data is static so labeling a fixed background object in a given camera would be quite lightweight
*Thread Reply:* Thanks a lot for the suggestion. Good idea to take advantage of static images to add more labels. 🙂
I have published the Bearcam Companion web application along with the latest blog in the series. In this post I discuss how to use AWS Amplify Hosting and GitHub to create an auto-deployed web application to a custom URL. The blog post is here. The Bearcam Companion website is here.
*Thread Reply:* Tagging some friends, in case y'all have any ideas and/or have read good papers on these topics: @Sara Beery @Lily Xu @Suzanne Stathatos
*Thread Reply:* Out of curiosity re: 1., have you experienced any problems with simply fine-tuning? Are you trying to achieve better results than that approach provides? More efficient training that exploits overlap?
*Thread Reply:* @Mark Goldwater: correct, I'm asking about ways to achieve better results than simple fine-tuning
*Thread Reply:* I think Alexander Mathis has some solutions for transfer learning. They are learning global models (they call them “supermodels”) for pose estimation of different species (e.g. a model for canines, one for birds) that can then be finetuned . All based on deeplabcut. For efficient finetuning, we generally use our software AIDE, with pretrained models that can be finetuned using some active learning scheduling to minimize the tuning cost. Check with @Benjamin Kellenberger if you want more details.
*Thread Reply:* not exactly the answer, but related: I am working on a fine-grained tree species classification paper that uses a set of nested hierarchical models, we found that pretraining on the flat model, then using that as a starting point for each hierarchical model was useful. Similarly, our 'bird' detection model has shown to be useful as a starting point for bird species classification.
*Thread Reply:* I second @Benjamin Kellenberger as the right person to ask on this. Generally what I've seen is that fine tuning works just as good as if not better than complex methods for transfer learning - though I could be in the same boat as you.
@Elijah Cole (Deactivated) has worked a lot on understanding the costs/benefits on label granularity I.e his paper here. He probably has good insight on (3) the hierarchical labels aspect.
*Thread Reply:* Thank you all for your responses!
@Ben Weinstein: how are you training the "set of nested hierarchical models"? And are those hierarchical labels different from the flat pretraining?
@Devis Tuia: sounds interesting - do you have any paper/publication you can point me to?
@Benjamin Kellenberger: everyone seems to be tagging you - would love to hear your thoughts on this!
*Thread Reply:* I don't know if this figure will be useful yet, but here is the conceptual figure from the draft. They are the same labels in both architectures, that's why I said it wasn't exactly what you were looking for. So training the flat model for all classes and then using that as the starting point for each of the models on the left.
*Thread Reply:* @Ben Weinstein: Is this understanding correct? You first train a classifier to predict the flat labels. Then you fine tune multiple instances of the classifier to focus of subsets of the flat labels. Finally, you combine these fine-tuned models in a hierarchical way. (But the individual models themselves are not hierarchical?)
*Thread Reply:* yes I think thats reasonable. When you say the individual models are not hierarchical, that interests me, if you mean that they have separate optimizers then yes. I've actually been really interested to see if anyone can show me a mixture-of-experts type hierarchy that is co-trained with a single combined loss function. Just for implementation, it feels confusing how to feed batches into such a network. We found it much easier to train separately the pieces and create separate dataloaders for each. Its definitely less efficient.
*Thread Reply:* @Chris Yeh, for the reference to AIDE: https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13489. For a reference to the supermodels, maybe @Valentin Gabeff can you provide one from Alex’s team (Alex is not in the slack channel)?
*Thread Reply:* Alexander pointed me to this pre-print: https://arxiv.org/pdf/2203.07436.pdf
The approach is a bit different as applied to animal pose estimation but the general design shares some ideas with what has been mentionned: Pre-train a 'SuperAnimal' model that performs well on different species, and then fine-tune it for a specific dataset (or in this case it would be fine-grained species) to improve performance. (see details about gradient masking and pseudo-labeling in the pre-print)
And no, Alexander is not here (yet?) ;)
*Thread Reply:* this is a really nice paper and something we would like to move towards for UAV-based object detection.
*Thread Reply:* You may be interested in our paper here: https://ojs.aaai.org/index.php/AAAI/article/view/17750, where we combine learning with hierarchical labels and automated reasoning over taxonomies.
*Thread Reply:* Hi Chris! Very late to respond here and I think others on this thread would be much more helpful on supervised learning tasks (I've focused much more on planning)
But in the planning literature, there may be some interesting ideas to explore: • learning diverse skills with RL (very related to transfer learning): this approach uses maximum entropy to train a base model, then use that pretrained model for specific downstream tasks. hierarchical RL to help solve more complex tasks https://arxiv.org/pdf/1802.06070.pdf • multi-task RL: uses "soft modularization" to train policy with shared parameters on multiple different tasks https://arxiv.org/pdf/2003.13661.pdf And one idea in the supervised learning world I really like is MIMO, multi-input multi-output. The idea is to use a single NN to act as an ensemble with N multiple inputs and N multiple outputs. Then you average the outputs as your final prediction. It's nice because it helps with stability (benefits from ensemble) and requires minimal extra storage or computation. There may be variations of this NN architecture that you could adapt to transfer learning skills. https://arxiv.org/pdf/2010.06610.pdf
*Thread Reply:* Hello G! I am working on an autonomous insect monitoring project in a research lab at Mila (Quebec). We are focused on lepidoptera in the moment.
Here is a recent paper that describes some of our current methods and partners: https://www.cell.com/trends/ecology-evolution/fulltext/S0169-5347(22)00134-3
And here is webinar that showcases some of the hardware projects in development: https://youtu.be/2Z8aG7qYAa0
I am currently working remotely from Portland, Oregon and I would love to connect with more folks interested in this topic on the west coast!
*Thread Reply:* https://www.bbc.co.uk/rd/blog/2021-04-winterwatch-artificial-intelligence-automated-monitoring
*Thread Reply:* Hi, I'm Robert, also from BBC Research & Development and in the same team as Graeme.
*Thread Reply:* Hi everyone! Also joining from BBC R&D, and also in Rob's and Graeme's team 🙂
I'm working this week on making/contributing to an open-source python package for aligning raw airborne imagery from UAV with the stitched georeferenced mosaic so that users can run machine learning models on high quality raw imagery and then try to place those detections/classifications into world coordinates. Many of you will have experience in these areas and I'm looking for any opinions/data on workflows. Given that the orthomosaic exists in a 3d or 2.5d space, every pixel in the raw imagery does not have a corresponding world coordinate. The larger the z dimension displacement, the larger the shift in the transformed location (orange box). This is in collaboration with https://github.com/UTokyo-FieldPhenomics-Lab/EasyIDP, which is a package that performs the reverse action, finds the location of raw imagery given a point in the orthophotos. I think this is a really underserved part of the workflow and something you can find many posts about in the assorted photogrammetry tools (pix4d, agisoft, etc), but no detailed workflow and example. All thoughts welcome. See this thread for technical details/code: https://github.com/UTokyo-FieldPhenomics-Lab/EasyIDP/discussions/44#discussioncomment-3579788
<a href="https://github.com/UTokyo-FieldPhenomics-Lab/EasyIDP">UTokyo-FieldPhenomics-Lab/EasyIDP</a>
*Thread Reply:* One of my colleagues specializes in UAV based 3d vision, and they're working through a lot of these problems on our noaa adapt program. I bet he'd love to talk your ear off about what they've done, if you want me to connect yall. Just let me know! This is outside my wheelhouse though lol
*Thread Reply:* happy to talk to anyone, it feels like the kind of task that has been solved (its more annoying than it is hard) but just does not exists in a repeatable way. I keep waiting for someone to just appear and be like, oh that's the 'export shape annotations' button, etc. But I've talked to developers from all the software companies and I haven't seen anything that suffices.
*Thread Reply:* Hey Ben, this sounds super relevant to a project we're running at FruitPunch at the moment.
*Thread Reply:* https://www.fruitpunch.ai/challenges/ai-for-trees
*Thread Reply:* I've worked on similar tasks where we used template matching with openCV to align overlapping geo-referenced DiMAC images - it's definitely not the fastest or sleekest option, but I think it's relevant. I'm not sure exactly how well it would work for trying to match ortho vs raw images, but it might be worth testing as a relatively simple baseline.
*Thread Reply:* When we did this sort of thing for Mars (matching rover ground imagery to orbit, different satellites, etc). Typically the process was to generate key points (SIFT) in both domains and then run some of robust matching and warping process. Which is probably the "draw the rest of the -- owl" bit... and I'm not sure how flexible the pipeline was for non Martian data...
Probably depends somewhat on expected localisation accuracy? But also if you're doing detection on imagery that was already used to generate the orthomosaic then don't you already have a warping function that's used to project points from each raw image into the ortho? (given the camera intrinsic/extrinsic info)
*Thread Reply:* We have a preprint (also in review) that describes our method for doing this (or at least similar) with drone video: https://www.biorxiv.org/content/10.1101/2022.06.30.498251v1 Basically use structure from motion to generate 3D georeferenced landscape models from some of the raw images and then use the resulting calculated camera matrices from those frames to project pixel locations into the georefernced 3D landscape space. We use local features in the raw frames to estimate the correct camera matrix even for frames not used to build the landscape model. The overal code we use is here: https://github.com/benkoger/overhead-video-worked-examples
NeurIPS workshop that explicitly requests submissions from real-world applications including conservation!
https://twitter.com/yoonholeee/status/1570461511797338113
}
Twitter
*Thread Reply:* "This workshop aims to convene a diverse set of domain experts and methods-oriented researchers working on distribution shifts. We are broadly interested in methods, evaluations and benchmarks, and theory for distribution shifts, and we are especially interested in work on distribution shifts that arise naturally in real-world application contexts. Examples of relevant topics include, but are not limited to: • Examples of real-world distribution shifts in various application areas. We especially welcome applications that are not widely discussed in the ML research community, e.g., education, sustainable development, and conservation. We encourage submissions that characterize distribution shifts and their effects in real-world applications; it is not at all necessary to propose a solution that is algorithmically novel. "
*Thread Reply:* • "Benchmarks and evaluations. We especially welcome contributions for subpopulation shifts, as they are underrepresented in current ML benchmarks. We are also interested in evaluation protocols that move beyond the standard assumption of fixed training and test splits -- for which applications would we need to consider other forms of shifts, such as streams of continually-changing data or feedback loops between models and data? "
*Thread Reply:* Hey Kamran! Thanks for your intro, your work with USFWS sounds so interesting! Is there a webpage for your research project on data collection from bald eagle nest cameras? Super interested in learning more as a fellow MS-CS student interested in birds!
*Thread Reply:* fellow u of c graduate here 👋
*Thread Reply:* While CV seems to be dominant in this slack, there is plenty of space for using other AI solutions for conservation applications! I'm part of a small, primarily RS-focused, team of geospatial analysts at a sm/med conservation non-profit. Happy to connect with you to chat about the work we do at CBI
*Thread Reply:* I’m also beginning to see a lot of applications that ML (and I guess CV) can do in ecology, and hoping to learn more - about both ecology and ML in tandem (I consider myself a newbie in both).
*Thread Reply:* Hi @Ethan Shafron, I am also interested in using ML and RS for carbon related questions (and the ecological and climate change implications), mostly storage atm, would be interested to hear more about your research and if you have any thoughts on modelling (quantifying) carbon storage in terrestrial ecosystems. Thanks!
*Thread Reply:* Hi @Declan - I've actually heard a fair bit about CBI! My partner works for the California Native Plant Society, and specifically works with EEMS - would love to hear about what you all are doing on the RS side of things though. Ofc there are plenty of non-CV ML solutions, I'm just off the heels of Sara's CV for ecology workshop, so that's what's been occupying my mind these days :)
@Rebecca - welcome! It's always cool to see "newbies" in spaces like these - if you're here then you're probably curious about a lot of different things, which is basically what drives this whole sub-field!
@Heather - There's a lot of stuff on Carbon modelling these days and it's hard to sift through it all to find what's relevant at varying spatial/temporal scales. Happy to chat about C fluxes and storage and hear more about what you're working on!
*Thread Reply:* Hi Ethan! I am very curious about this side of things. Do you think I would be able to PM to chat more with you? I have a few questions about how to grow and position myself to be on the conservation side of things, and would love to chat about an idea that has been brewing.
*Thread Reply:* Hi Josh! Fellow monkey enthusiast here! I'm finishing my PhD on global primate biogeography and conservation. I suppose the first question is, what is your favourite monkey?
*Thread Reply:* Hey @Heather, fellow Canadian as well i suppose (guessing from your email)? And wow that sounds really cool!! I would love to hear more about your studies 🙂 and I love squirrel monkeys because of how mischievous they are for sure haha, but I am fascinated by most of the platyrrhines
*Thread Reply:* Super cool that you've done so much while still in high school, especially the NASA project that sounds amazing!
*Thread Reply:* Yes it is extremely impressive!
*Thread Reply:* Very interesting research! I also looked at the ID Trees project - super intriguing to me since I want to look into trees and forests!
*Thread Reply:* I would love to read your master’s project if you are willing to share, @Adam Noach!
*Thread Reply:* I'm in that awkward middle ground of extending it into a paper - but when it's cooked up ( & if it's appropriate and welcome) I'd be happy to share it here. I can see it's already been mentioned but DeepForest will be up your alley if ID Trees was!
*Thread Reply:* Got it! i will look at that.
*Thread Reply:* Hmm when I have mastered Python in entirety, I will play around with this package.
*Thread Reply:* Hi Heather! Glad to see you here 😄
Hi everyone. Can we do a big round up on current drone's that teams are using. We are running out of time to update from our DJI's to get up to date with the federal drone list: https://www.diu.mil/blue-uas-cleared-list. A general desire is • Quadcopter for heavy wind conditions landing on a boat (We have fun videos of staff diving into the alligator heavy everglades to recover vertical take off drones) • Under 30k • PPK accuracy for geotagging each image • On the US federal blue list We own several DJI Inspires and a Wingtra One (Gen I).
*Thread Reply:* Hey ben - this might be a question to also drop in to the WILDLABS dicusssions and tag to our drones group so you can get feedback from the wider conservation tech community. https://wildlabs.net/
*Thread Reply:* Hi @Pietro Perona there is also a #camera_traps channel as well where this could be good to ask!
*Thread Reply:* We have great nightime pictures of quick animals with bushnell trophy no glow. Seems the new version is even better than the ones we bought in 2020 https://www.bushnell.com/trail-cameras/standard-trail-cameras/core-s-4k-no-glow-trail-camera/PB-119949C.html
*Thread Reply:* Looking forward to those NE italian night shots!
*Thread Reply:* why not under #new_papers ?
*Thread Reply:* It could be under new papers…it’s just that not all new papers will introduce new datasets. Would be much easier to have a channel with only links to datasets old and new.
*Thread Reply:* You're free to make any channels you think are useful!
Hi everyone! I'm a researcher at RISE, the Swedish research institute. I have a PhD in computer science, and am now heading the deep learning research part of our center for applied AI. We have some exciting projects related to biodiversity and modelling of earth and water. I was just invited to this Slack by @Devis Tuia, whom I just had the pleasure to have invited to our seminar series, RISE Learning Machines, and who just gave a great talk there. We are looking for master's students for a couple of projects this spring, including Deep learning for detection of coffee berry disease and Aerial View Goal Localization with Reinforcement Learning. Great to be here!
Really neat to see Esri releasing a bunch of pretrained models relevant to this Slack, including models based on data that folks on this Slack have helped make available:
That includes a tree detection model (based on @Ben Weinstein's NEON tree dataset), a bird detection model (based on @Benjamin Kellenberger's Aerial Seabirds West Africa dataset), an elephant detection model (based on the Aerial Elephant Dataset), and a bunch of other interesting ones that aren't quite so on-the-nose conservation-focused, but are relevant and interesting.
*Thread Reply:* this is cool, but also kind of terrifying, there is alot of improvement to be made on deepforest and a company with ESRI's capability could really make it better. I get nervous when it gets just passed around as a 'tree detector' without all the docs associated with deepforest about all the time it doesn't work/could be better.
*Thread Reply:* I think @Pietro Perona is the best bet here
*Thread Reply:* We are looking for the data - usual problem w switching servers. Please also contact Jan Wegner at ETH
*Thread Reply:* Oh I'm super-interested. I had no idea ESA was in Portland next year. My dream is to get all the benefits of in-person conferences without actually going anywhere, which I'll roughly define as "without leaving Seahawks territory", and you're telling me I can have TWS in Spokane followed by ESA in Portland, and meet lots of conservation folks, and never really have to go anywhere? Sign me up!
*Thread Reply:* I live in Portland and will submitting a proposal, I welcome everyone to join me. I'll post it here. I'll probably host a dinner too.
*Thread Reply:* Something like, "Automated ecological monitoring using Computer Vision: Where are we going? How do we get there?"
*Thread Reply:* Thanks Dan and Ben! I wonder if we could get a couple of sessions proposed?! In addition to Ben's broad topic area, maybe something on Application of Deep Learning in Wildlife and Fisheries? Organized oral sessions are 6 talks/Symposia are 4 talks (longer; broad topics)/Inspire Sessions are 6-10 talks. Ben- what category are you thinking? ...Conference theme is "for all ecologists" - meaning both academics and us applied folks 😀
*Thread Reply:* I am also in Portland and would love to attend if not present as well. I could contribute some slides on the state of automated insect monitoring, or perhaps do a shorter "Inspire Session" on that topic. I would love to rally support for getting a network of insect monitoring stations setup in the USA, or at least up and down the Pacific Northwest
*Thread Reply:* @Ben Weinstein that sounds like a fantastic proposal - I'd be interested!
*Thread Reply:* I'm also definitely interested! I've never been to an in-person ESA, and I share @Dan Morris's enthusiasm for meeting lots of ecologists and conservationists without needing to go too far!
*Thread Reply:* Yes to all of this! @Ben Weinstein I’d be game
*Thread Reply:* For those planning to attend, I'd also like to invite you to visit my new computational imaging lab at Portland State University in downtown Portland!
(One of my research thrusts is in high speed/low light image sensing under resource constrained scenarios. I believe this can have applications in ecological monitoring esp situations where conventional vision sensors+algorithms fail.)
*Thread Reply:* For background, it is the largest ecology meeting in the US -- precovid- upwards of 4,000 people regularly attend. ESA is also one of the more high-tech ecology conferences and a good place to see the next big ideas in ecology as a whole (overarching plants, insects, wildlife, fisheries, landscape ecology, land cover mapping). Some of these topics can be rare at other conferences. Looks like we should have interest for at least 1-2 sessions...
*Thread Reply:* Hi all! I'd like to keep track of this thread related to the ESA conference in Portland this year, and everyone in the PNW who are generally interested in staying in touch! The messages in this Slack workspace are suppose to expire in 90 days. I've created a new channel <#C04L722GH26|region-pnw>, which doesn't solve the 90 days problem, but might help us stay familiar with each other.
*Thread Reply:* Our session (@Dan Morris, @Sara Beery, @Emily Lines, @Tessa Rhinehart, @Kakani Katija), "The future of ecological monitoring is collaboration with Artificial Intelligence", was accepted.
*Thread Reply:* Oh fantastic! I am part of another session proposal focused on insect monitoring. Still waiting to hear back, but I look forward to the event regardless. Perhaps an AI for Conservation happy hour is in order?
*Thread Reply:* Hi Eric! Nice to see you here! (I was a DS at Sfix)
*Thread Reply:* Hey Eric, great to see you here!
Excited to share with the community our first peer-reviewed FathomNet publication in Scientific Reports: https://t.co/G1gLUdNDde. FathomNet is a multi-institutional multi-individual effort to build a global labeled image database for underwater life (and other objects). Seeded with data from MBARI, NOAA, and National Geographic Society, we look forward to data contributions and expertise from a broad user community. Check it out, share it with your networks. FathomNet can be accessed at www.FathomNet.org, follow us on Twitter @fathomnet, and we have a python API at www.GitHub.com/FathomNet. Looking forward to receiving your feedback!
*Thread Reply:* does fathom net come with a detector in the python client. Like if I have an image can I go
import fathomnet
m = fathomnet.model()
predictions = m.predict_image("my_image")
predictions["class"]
<awesome_sea_snake>
*Thread Reply:* No it doesn't. You can check out our model zoo at https://github.com/fathomnet/models and more details on the python api are at https://github.com/fathomnet/fathomnet-py
*Thread Reply:* Congrats! Super exciting
*Thread Reply:* The model zoo is very cool! I have no particular reason to want to detect benthic organisms, but who am I to turn away a well-documented model zoo? So FWIW I was able to download and run the "MBARI Monterey Bay Benthic" model, it was very straightforward.
*Thread Reply:* Awesome. Would love to get your input @Dan Morris on how we can make it better (e.g, standardization, etc).
*Thread Reply:* My only off-the-cuff recommendation would be to include images on the model zoo README that conveys "this is the gestalt of the training data for each model"... that makes it a lot easier for a new user to judge whether the angle/color/murkiness of their images is close enough to the domain of the model to expect reasonable results, and IMO no amount of verbal description can capture that. But once a user decides "yes, this model is reasonably close to my data", the models look well-documented.
*Thread Reply:* Yeah, we’ll have to think carefully on that. Totally agree.
*Thread Reply:* how close are we to higher order taxonomy detection for these type of images. Fish/Jellyfish/Octopus/Crab. How deep the taxonomy can we go?
*Thread Reply:* I am imagining that if there was some live underwater feed and we wanted to partition the images to the relevant annotators/experts. When you see a seastar send it to seastardude@gmail.com, etc.
*Thread Reply:* This is really cool! We are quite busy with underwater imagery here at ECEO, I will definitely share that with my students!
*Thread Reply:* @Ben Weinstein, check out the model zoo and the MBARI benthic object detector. We're already pretty close for high-level morphological classes (of course could be better); doing the same thing for midwater animals now too. The FathomNet taxonomy goes to species. Yep totally! The community mods (eg notifications, etc) are currently underway for FathomNet. Should be released by Spring. 🤞:skintone4:
*Thread Reply:* I've been building some higher order marine organism detectors that we're using on AUVs in the field that are "working" to some degree, partially using some of the fathomnet dataset. Happy to share/chat.
*Thread Reply:* Great! @Levi Cai, you should contribute your models to the FathomNet Model Zoo!!
*Thread Reply:* Hi, I want to know are there any recordings for these talks?
*Thread Reply:* Dear @Kangyu Zheng, all workshop slides from all speakers will be up at the weekend on the website. We did not record this 1st workshop to avoid any GDPR issues, keep it very low key and allow for just a single public link for people to join. The next edition of the workshop will provide recordings and hopefully full proceedings. Best, Tilo
*Thread Reply:* Out of curiosity, what species does the model cover? Also, do you know how quick the on-board processing is? Like how many seconds/image type thing
*Thread Reply:* @Carly Batist we are training the classifier on deer, elk, moose, domestic livestock (cattle, sheep), wolves, coyotes, mountain lion, bear, rabbits, humans, vehicles, and a smattering of other things (domestic dogs, badgers, bobcat, wild turkey, etc). For inference, once the models are loaded and initialized it takes just under 1 second per image on the Jetson Nano to run through MegaDetector and a little less than that to run through the classifier model. So maybe inference at 1.5 seconds per image. We are seeing a full cycle of wake up, pull images off trail cam, detect/classify, transmit via satellite, sleep typically takes about 5 mins.
*Thread Reply:* This is fantastic. Congratulations on making an end-to-end system, and with a satellite connection! I look forward to looking at the design and code when you are ready, and I would be happy to test them. What did you use for the camera?
*Thread Reply:* Thanks @Michael Bunsen. It currently works with a Moultrie M-50i camera. As built, it could work easily with other cameras that support automated "USB Mode" where it presents as a USB drive when it detects power on its underside USB port.
*Thread Reply:* Very cool! There are a number of related efforts (e.g. the Conservation X Labs work announced on another thread), but AFAIK all the other work like this is targeting a very general-purpose device, which will definitely be necessary to shift the field in this direction, but I think we'll also need some successes where projects like this make something work end-to-end for the team that's building it, starting in a small (ideally n=1) number of ecosystems, where you can really kick the tires on your own end-to-end process, and inform all of the work on general-purpose devices. Excited to hear how field tests go!
*Thread Reply:* @Luke Sheneman looks super interesting! We are also using the Jetson Nano for a recent camera trap prototype and have some experience with optimizing power consumption & MegaDetector models, so let me know if I can be of any help!
*Thread Reply:* Thanks @Timm Haucke! I took a glance at your paper and it looks really interesting. We'll assess real-world performance and battery life in the field soon, and I am sure I will have some questions!
*Thread Reply:* I have about 20 jetson Nano (4GB, A02) dev kit left and could sell them, since we switched to the NX. If any one is interested DM me. Currently they are just collecting dust.
Since availability of the dev kit is super bad, this might help some of you. - (Hopefully i am not spamming this thread.)
*Thread Reply:* I don't exactly have an answer for you, but I know some colleagues at the Point No Point Treaty Council:
...who are using technology (i.e., the kind of technology we talk about on this Slack) for wildlife surveys in the area governed by the Treaty of Point No Point:
https://goia.wa.gov/resources/treaties/treaty-point-no-point-1855
...which protects fishing and hunting rights for several tribes in Western Washington. If you email me offline (agentmorris@gmail.com), I'm happy to make an introduction if that would be helpful.
*Thread Reply:* I know the young folks are already involved in learning the names of and identifying their wild and natural species.
Would curated data collection on the ground or cameras set up to survey native species be of use to anyone in your network?
Perhaps passive collection as the young people survey their land is helpful? If so, I would be very happy to see how their “species learning” could serve this conservation community group as well.
*Thread Reply:* @nyakundi lamech was involved in a youth project that does exactly this. But I don't know how valuable comments from a project outside of the US would be to you?.
*Thread Reply:* Always valuable thank you! Please message me I would love to learn from this expertise and experience
*Thread Reply:* LILA BC is a good place to look for camera trap datasets.
Are you looking for temporal shifts at a multi-year-long level, though, if you’re looking for population changes?
*Thread Reply:* Thanks @Suzanne Stathatos! I'm mostly looking for classification datasets with trend and/or seasonal (or both) change in the class probabilities over time. For example, when studying the Snapshot Serengeti dataset, I noticed that the estimated population of Wildebeest is close to zero during dry months because they migrate out of the Serengeti for locations where there is grass and it when starts raining in the Serengeti, they migrate back because the grass is more rewarding. So my study of the animal population over time is to find those species with predictable changes.
*Thread Reply:* Very cool!! I think Snapshot Serengeti is the longest-running accessible, public dataset with images from camera traps, but iNaturalist could be another interesting modality to investigate? There you also have the added complexity of opportunistic sampling, but in aggregate changes are captured over time
*Thread Reply:* I would also check out ecotaxa. They focus largely on plankton classification with multi-year observations using a suite of underwater and in-lab bright field imaging tools.
*Thread Reply:* Thanks @Sara Beery and @Kakani Katija. I'll check them out
*Thread Reply:* Nice, I live in Leiderdorp, do you live in Leiden area? Might be nice to catch up?
How did they count the frogs? https://www.bbc.co.uk/news/science-environment-63206140
*Thread Reply:* Yeah that's a great a question. As they describe it, it sounds like these authors base their estimates on what we call dynamic occupancy models, ( https://www.pnas.org/doi/10.1073/pnas.2123070119) Such models essentially seek to model the probability of detection for some latent occupancy, (see methods section and appendix https://www.pnas.org/doi/suppl/10.1073/pnas.2123070119/suppl_file/pnas.2123070119.sapp.pdf). such models are typically bayesian hierarchical models estimated by MCMC -- they provide their code at the end of the supplement).
I think this area is ripe for advances from the ML community but doesn't get as much attention as it could! Numerical constraints often mean that these models must ignore lots of factors that probably also influence probability of detection, as well as generally ignoring ecological factors like whether like to group together or establish separate territory (i.e. so that observations are not properly independent) etc etc. Because there is no perfect census method to compare to, it's rather difficult to say how well current methods of counting actually work.
Hey Everyone, I’m new to this community. My name is Titus and I’m the VP of Strategy and Head of Computational Sciences at Colossal Biosciences. My background is in AI+Genomics but my team is working across genomic, audio, and imaging methods for conservation and de-extinction. This community is great and looking forward to chatting with people as the opportunity comes up! If anyone ever needs anything or wants to chat, shoot me a note.
*Thread Reply:* Hi!
Maybe it is far afield but one area where we are really lacking in preservation efforts in the biodiversity created by diverse diets, food practice and local ecologies.
I’d be thrilled to discuss further if your team has been thinking at all about how to preserve and maintain vaninshing human/health associated microbes and plants :)
I am a human microbiome PhD student at UCB but also part of their nsf digital transformation of development cohort which helps PhD students center their questions in translational efforts that serve humanity, particularly vulnerable groups (people, species etc).
I am looking for better ways to expand the questions we ask around the human biome to push preservation of diverse cultural food practice and diverse edible species.
If any of that helps I would be very happy to talk more (at least on the bio side of things).
I am still teaching myself the computational side!
Feel free to email Giana@berkeley.edu
*Thread Reply:* Hi! I am not involved in the purely plant side of this, but I know many efforts are under way to think about conservation of human internal ecological biodiversity.
It’s a really difficult topic because it’s not easy to fund collections in non US populations and also there are limited frameworks for ethical and human/culturally centered approaches to storing and using that data.
Happy to talk more if any of my information relative to the human biome can help!
*Thread Reply:* Hi Giana, this sounds very similar to some of the work being done by Propogate- much of their tools do this with a focus on agroforestry as a method for sustainable economic development, conservation and carbon sequestration, you might find their analytics tools to be exactly what you're looking for. Their tools utilize multiple input layers to assess soil quality, hydrology etc to outline which species might be best to plant. Keep in mind theyre not doing this directly with AI but with pre-generated spatial layers. Their resolution is pretty decent as well. Happy to put you in touch with one of my friends who works there if its of interest. https://propagateag.com/
Best of luck!
*Thread Reply:* My goodness that would be such a gift! Thank you so much I would be so grateful.
My email is giana@berkeley.edu :)
*Thread Reply:* I'm thinking about getting involved right now, dm me if it looks interesting to you
*Thread Reply:* I just signed up the other day! I haven't gotten around to watching the introduction videos yet, as there is a lot to take in and some of it is honestly beyond my comprehension at this point. I'm definitely interested though and would love to team up with someone : )
*Thread Reply:* Hi Sicily, I was in the 2022 cohort and would be happy to chat! Send me a DM 🙂
*Thread Reply:* Hi Sicily! I was also in the 2022 cohort and happy to chat if you want (also hi Rowan!)
*Thread Reply:* Hi both! Thanks for the message this sounds great. Will PM you!
*Thread Reply:* You can have a look at the work done by iNaturalist - the seek app,
*Thread Reply:* Hi Phuc, feel free to send me an DM/email, happy to make some time to talk if you're interested! Key features for turtles are mostly carapace and plastron shots. Side photos of the head being the second most important feature. I also strongly recommend trying to develop a mask/segmentation model to extract the shell
*Thread Reply:* Thanks @Valentin Ștefan for the insights! I have used Pl@ntNet before and was impressed by its ability to recognize in Vietnam. I totally forgot about it until you mentioned. I will take a look at their technology. I also used YOLOv5 to localize the position of the turtle in an image and it generalized quite well. But I think I will use different networks for the classification task because I want the model to return many predictions so that users can verify them. But still the guide from YOLOv5 team is very informative and helpful!
*Thread Reply:* Hi @Alexander Robillard, thanks for your note, I have DM-ed you for further discussion 🙂
*Thread Reply:* Note that, with the new release of YOLOv5 - v6.2, you can also do image classification not only localization. I think you can tell the model to return several predictions (possibly with the --conf-thres and --iou-thresin the detect.py script. If you explore further with YOLOv5, let me know as I am also interested in how it performs and how can the weights be deployed as an app on a phone. At the moment, I am unsure about how fast will it run on a phone and I do not have the coding skills yet to build an app. I presume the nano weights version would be a better option
*Thread Reply:* I didnt know that they have classification versions. Thanks a lot. I will test these models out.
*Thread Reply:* Imo, u can just take the photos with the best quality, then lower its quality during training as augmentation. Also, i dont see why u should stick to yolov5, u can just do yolov7, any other new SOTA ones. And yes, u should take into accounts all those data characteristic. Hey dm me, I'd love to discuss and help out
*Thread Reply:* @Anh Quoc Nguyen I've DM-ed you!
*Thread Reply:* Is this for a specific animal or all species that might be present in the study area?
*Thread Reply:* Do you mean by some sort of light/sound stimulus?
*Thread Reply:* This one in particular is for Large Cats
*Thread Reply:* I have mega tons of data that is from the side but very limited data on direct facial images
*Thread Reply:* (especially because much of the data is at night)
*Thread Reply:* I see much of the Snow Leopard cats take a min to look directly at the camera.. I would love to know how.why
*Thread Reply:* Not sure if you can use images for this (may need video because it happens so fast), but mountain lions often look directly at cameras before fleeing when the cameras are paired with audio playback of human voices... https://www.washingtonpost.com/news/speaking-of-science/wp/2017/06/21/mountain-lions-are-terrified-by-the-voices-of-rush-limbaugh-and-rachel-maddow/
*Thread Reply:* If you're restricted to using just images, and no video, I would imagine you'd need a scent lure or something they are more likely to investigate for a longer period of time.
*Thread Reply:* Interesting, thank you. Rush L. WOuld scare the hell out of me too ;) I am looking for noninvasive methods. We are also collecting video and I was wondering about the use of scent. DO you or anyone have experience using it?
*Thread Reply:* Yes, but I would have to dig it up from archives of messages, and am running to a meeting now. I will make a note to come back and respond!
*Thread Reply:* I've only owned cheap consumer camera traps, but I'm curious whether anyone knows about the bandwidth of the IR flash on research-grade vs. consumer-grade cameras. I wouldn't say it's "giant red light" on my cheap consumer camera trap, but it's close: you can definitely see visible red during the flash, and every coyote walking by the camera looks right at it. I would guess the flash is much narrower in bandwidth on serious cameras, but I have no data to back that up. But if that's accurate, maybe a solution is: use a terrible camera?
*Thread Reply:* [doesn't help you during the day]
*Thread Reply:* Of course, sound will be more effective in attracting attention (I assume), but it sounds like you're trying to walk the fine line between attraction attention and deterring presence, and maybe "unintentional red flash" hits that sweet spot.
*Thread Reply:* https://onlinelibrary.wiley.com/doi/full/10.1002/ece3.8536
*Thread Reply:* sounds like they played a recording of a juvenile puma call plus light flash to get a mugshot
*Thread Reply:* baited camera traps can work but then your observations are beginning to interfere with their natural behaviour so depends a bit on the overall objective
*Thread Reply:* I've only been around this space for a short while, but I've seen the discussions that you led about standards and how "not to re-invent the wheel", especially when it comes to processing images. I think that's another one to add to the list: we have many tools but nothing that has been widely embraced (except maybe for megadetector?).
*Thread Reply:* Wow, you've added tons of features here! Out of curiosity: it's rare for an ecologist to need to add/manipulate bounding boxes for analysis purposes, so I assume that you and/or users are using that functionality to either fine-tune MegaDetector or train a new detector... is that right? Can you share anything about what the edited boxes are being used for?
*Thread Reply:* A user wanted to use MD to kickstart his annotation process, so that he didn’t have to manually draw all the boxes. He would then only have to change the label to the species (and fine-tune the results, if needed). With this reviewed and annotated data he would train his own site-specific model. That’s why I added the feature of converting MD’s output.json to individual .xml files and getting rid of the labelImg installation hassle. I agree with you that most users won’t use this feature, but, nevertheless, it’s possible :)
*Thread Reply:* Great work, thanks for sharing! If a user already has some of the dependencies on their system, can this tool use them instead of installing everything from scratch with the batch file?
*Thread Reply:* potentially of interest for future developments: over the summer I worked with others in the DeepLabCut team integrating MegaDetector and DeepLabCut. You can check the prototype at the following HuggingFace space
*Thread Reply:* Ahhh this was exactly what i needed in July. I love this! Thank you so much!
*Thread Reply:* @Pen-Yuan Hsing The install files search for a working installation of anaconda and git for windows. If present, it will use them. The rest (gits and models), however, will be downloaded from scratch and stored in a hidden folder. This has a few reasons: I) I want the gits to be checked out at a specific time to avoid conflicts with newer commits, II) it will slow down the installation and opening of the app significantly, and III) if the user then adjusts a directory somewhere down the tree, it will error.
@Sofía Miñano very interesting! Might be worth incorporating in a future version of EcoAssist too 🙂
*Thread Reply:* This is great! I've been contemplating doing this, and it looks like you've done a great job here.
*Thread Reply:* AWESOME! Thank you so much for your work! Some Iberian lynx field technicians are starting to use it, and they are fascinated!! They are very happy!! One question...about the use of custom yolov5 model, there are any consideration to take in account? (before to think about a train a new one?) Thanks so much! @Peter van Lunteren
*Thread Reply:* Hi @Anton Alvarez, Thanks for your message. Always good to hear that people can use it for conservation 🙂 EcoAssist can run custom yolov5 models if they are retrained from the MegaDetector model using transfer learning. For example, if you find that MegaDetector is not great at recognising a certain species as “animal”, you can retrain the model and add some labelled data of the cases you want it to improve on. Additionally, you can expand on the three default classes MegaDetector uses (“animal”, “person” and “vehicle”) and retrain the model to detect custom classes (e.g. “Iberian Lynx”). In that case, if you add classes, you’ll need to adjust run_detector.py and separate_detections_into_folders.py too. If you need any help with that, let me know!
*Thread Reply:* Thank you very much for clarifying everything, it was to have things clear before embarking on the training of the model, (i need to allocate my time or budget) if I have any questions I will contact you. And thank you very much (again) for the new EcoAssist v2.1 version!
*Thread Reply:* I finally got a chance to try EcoAssist today, and not only am I super-impressed (it worked great on my Windows PC, even using the GPU correctly), the installer scripts you've written are absolute works of art. MegaDetector aside - in fact, conservation aside - you have a really neat approach to packaging and deploying Python code in general for users who have never heard of Python.
Batch file nerds all over the world will be in envy of what you've done in that batch file. I bet somewhere there's a Reddit forum called "things you didn't think you could pull off in a batch file", and if there is, you should totally post there.
The fact that you even did all the magic stuff to allow accelerated inference on M1 Macs - still with a one-click deploy! - is icing on the cake.
*Thread Reply:* Do you mean like Simpson’s paradox? ( https://en.wikipedia.org/wiki/Simpson%27s_paradox )?
WILDLABS’ annual State of Conservation Tech Survey is open! https://www.linkedin.com/posts/wildlabs-community_tech4wildlife-activity-6994417296681160705-t3mp
*Thread Reply:* (Extra info: Preference for an early-stage startup, but mature enough to have at least ~5-10 engineers already. Her background is 6+ yrs in Lead & Senior Software Engineering roles at an e-commerce company, working with React, TypeScript, Gatsby, Node.js, and AWS.)
*Thread Reply:* The Climate Change AI newsletter is also a good place to look for job opportunities!
*Thread Reply:* I know people who are or were at Jupiter Intelligence. Looks like they might have some openings: https://jupiterintel.com/jobs/
*Thread Reply:* We’ve got some job boards on the Conservation Tech Directory! Including the previously mentioned Climate Change AI, which I second as a great resource!
*Thread Reply:* There are also 2 slack channels that I know of dedicated to particularly climate-related things: • ClimateAction.Tech • WorkOnClimate
*Thread Reply:* Thank you so much for your suggestions @Sara Beery @Toryn Schafer @Carly Batist and @Suzanne Stathatos, we really appreciate it!
*Thread Reply:* There is a #jobs channel! Cheers
*Thread Reply:* whoops let me see if I can move it, thanks.
*Thread Reply:* Deleted as wrong channel.
*Thread Reply:* This brings you directly to the PhD projects description: https://wilddrone.eu/recruitment/
*Thread Reply:* Hi @Devis Tuia, I am not a PhD student but I am highly interested in this work for my master's thesis. Thanks for sharing. I will likely reach out to the supervisors to establish correspondence with them. Is there anything I should know before contacting the advisors for DC6, DC8, or DC10? Thanks again!
*Thread Reply:* I would say, check the different project details on wilddrone.eu so that you can have a clearer idea of what they are about and then contact people. We are hiring at the PhD level, but maybe the single institutions have capacity for extra master projects!
*Thread Reply:* Congrats, love to see it !
*Thread Reply:* I appreciate that Devis. I'll reach out to the individuals to discuss. Personally I probably won't be able to move for these opportunities at this time, so I'm mostly interested in just establishing contact and sharing our respective projects, perhaps collaborating on a paper along the way 😄 . Thanks again for the info!
*Thread Reply:* did you ask to Gert Koostra at WUR?
*Thread Reply:* @Helena Russello works in his group in case you need more info
*Thread Reply:* This postdoc sounds exciting, what megafauna are you looking into?
*Thread Reply:* Hi Tiziana, sorry for the slow reply (slack didn't notify me of the updates in this thread). The WuR research group working on drones is working in African game parks so expect anything from antelope to elephants. In the Netherlands they are aiming to count geese and deer using drones. For this small pilot we want to map the synergies between 3 related topics and investigate a unified approach. We probably found a person to work on the pilot, not 100% sure yet. PM me if you want to connect 🙂
*Thread Reply:* Not directly addressing your job posting, but I notice you’re in Montreal. You should check out VT Tech Jam next year! A short drive to your south in Burlington, VT. https://techjamvt.com
*Thread Reply:* Hey awesome, thanks @Anselm Bradford! We are actually partially funded by the Vermont Center for Ecostudies so already have one foot in Vermont!
*Thread Reply:* Oh neat! Check out https://www.caryinstitute.org/about too. Maybe relevant resources there also.
*Thread Reply:* Out of curiosity, why the extra effort to use IR-reactive paint, instead of just visible paint? Is this a case where you expect that paint within the visible spectrum of the animal (or other animals) will have a behavioral/survival impact?
*Thread Reply:* Just guessing, but it might be more reliable to blob count in a spectrum where you're unlikely to have false positives. I'm interested in hearing the reason too though (along with any potential paint recommendations)
*Thread Reply:* We had some Ideas to use glitter as unique finger print for bees. Just spray them and the glitter pattern will be unique for every bee. The concept worked as a proof of concept. But in practice capturing the patterns on moving bees was not as easy, so we just glued markers on them. (MNIST for bees). Even though it was way way way more work, then just spray glitter on bees.
Side note: I used UV reactive paint for another use case, trace the movement of bees pollinating and could share some details if you are interested.
*Thread Reply:* @Dan Morris, @Frederic , @Nick Giampietro - Yeah, it is for AI-controlled automated mark recapture in a controlled dark environment. We also don't want to impact normal predation by making the animal stand out more. The perfect "paint" would be IR reflective/absorbent but also otherwise invisible. Such a thing might not exist?
Although it would be an added bonus, we don't necessarily need to identify unique individuals, just recognize if we've seen the animal before. Frederic - I would be interested to know what paint you used for your bee work. Your glitter idea for bees is fascinating, but I can see how that might be challenging in practice!
*Thread Reply:* Yeah, it's probably hard to find a pigment that reflects IR without also reflecting more reds as well, since spectral radiance tends to be some number of smooth curves with a peak at a given frequency and, well, IR is close to red.
This article has some useful info, and even some pigments that look close to your requirements... Can't speak to whether they would be safe to use though
https://www.pcimag.com/articles/102920-ir-reflective-pigments-a-black-rainbow-of-options
*Thread Reply:* If you find a way to mark the animal without it being visible to a non-IR camera, then you will be able to make ground-truth data for detecting individuals from normal RGB images as well.
*Thread Reply:* @Carly Batist @Taiki Sakai - NOAA Affiliate Might have some pointers 🙂
*Thread Reply:* Hi! I do PAM with lemurs in Madagascar as well as some soundscape indices/analysis type stuff. I use ML models to detect lemur calls, with the huge help of my computer science expert collaborators @Emmanuel Dufourq and Lorene Jeantet! I have some materials on my website regarding PAM, including a list of good papers to look at for different use cases which might be helpful for you. You may also want to check out the Conservation Tech Directory to search for projects, tools, organizations, etc in that space (filter/search by audio, passive acoustic, etc). Happy to chat further over DM or hop on a zoom if you’d like to talk further!
*Thread Reply:* @Holger Klinck works a lot with audio as well!
*Thread Reply:* also, Dan Stowell is working on audio and biodiversity
*Thread Reply:* Hi Nicolas! Our lab uses passive acoustics to study marine mammals. Audio data is used a lot in our field since sound travels so far under water (among other reasons). AI/ML is definitely being used more and more, Dan Stowell's name has already been mentioned here but if you haven't read his review paper on bioacoustics and deep learning that's an excellent starting point.
Happy to chat if you have any marine mammal questions!
*Thread Reply:* Such a warm welcoming! Thanks a lot to all of you for the info, I am going to look into detail at the resources you shared and will keep in touch 😊
*Thread Reply:* @Nicolas Arrieta Larraza, see my publication in the main channel posted today; we do have tons of audio analyses to do and potential future opportunities.
*Thread Reply:* https://twitter.com/mmbronstein/status/1597656746767822848
*Thread Reply:* Hi Nicolas, if you have any specific questions, please reach out. We are working on a bunch of marine and terrestrial projects. Here is one of them: https://birdnet.cornell.edu/ We also released a Python package for machine listening: https://shyamblast.github.io/Koogu/en/stable/ Cheers!
*Thread Reply:* Hey Tony!! @Antonio Ferraz I know Cal Poly San Luis Obispo didn't make the NASA/JPL cut, but I know an ecologist here who might be able to do what you describe. I have a colleague at Cornell who might know some bird folks there. Let me know if you'd like to pursue any of those opportunities. Hope all is well.
*Thread Reply:* hey, how are you? yes, let’s reconnect shortly
*Thread Reply:* Hey man, doing good, busy busy. How about yourself?
*Thread Reply:* Let me know if you want to connect. im sure you probably found your biologist by now.
*Thread Reply:* Haha, yes, have been working on this for years. Also with @Subhransu Maji. What kind of birds? We have an ongoing project for detecting and tracking swallow roosts (AAAI, bioRxiv). Swallows (also bats) form the most distinctive patterns on radar, but many other species have related behaviors that also show up on radar.
*Thread Reply:* Thanks. This is all very preliminary, as part of a future grant proposal, so just trying to sketch out what is possible. Briefly, the everglades restoration act is spending about a billion dollars to reform large waterways and canals in south florida. One of the metrics of success is wading bird colony population numbers. These are large aggregations of nesting birds, anywhere from several hundred individuals to 25,000 megacolonies. We cover large areas (>1000 sq km, the WCA1,2,3 in the map) using piloted aircraft and drones, and have been looking for an early warning system to be able to locate colonies so we can send aircraft there. We have solid machine learning models for the optical data and 30 years of historic data that we could associate with locations of colonies that could serve as potential supervised labels. My question with nexrad is 1) What kind of spatial resolution can we expect in the 'legacy' (not dual-pol) stations. KAMX is right next to our sites. When we see dots in the nexrad, is that an individual bird? a flock? What kind of target size is it picking up? 2) i've seen this mostly applied to martins and other species with a single very large signature at dawn. The herons/egret/ibis are larger, but the pattern will be more chaotic as they come and go from the colony many times a day. Does this seem feasible? I'm imagining a workflow for detection colonies of high activity 1) getting nextrad data, 2) traditional background subtraction to remove noise (why is there always a ring of responses around the detector?) 3) look for patterns of presence in the biological reflectance range (i've read 20-30db with low cross-correlation ratios from https://esajournals.onlinelibrary.wiley.com/doi/10.1002/ecs2.1539). 4) temporal tracking of objects to perform optical flow, 5) flow patterns coming in an out of potential colonies using a recurrent neural network (not enough temporal coverage?). Happy to jump on zoom if its easier. I'm playing around here https://github.com/weecology/everglades_radar/blob/main/Download.ipynb
<a href="https://github.com/weecology/everglades_radar">weecology/everglades_radar</a>
*Thread Reply:* Super cool. The scans in the notebook look somewhat different from some of the roosts we have been looking at (they look more like nocturnal migration). You could possibly use MistNet to separate rain from biology that works on non-dual pol data (e.g., cross-correlation is a dual-pol product). https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13280
*Thread Reply:* interesting. I see the cross_correlation on this station, even though it is labeled legacy.
radar.fields.keys()
dict_keys(['reflectivity', 'velocity', 'differential_reflectivity', 'cross_correlation_ratio', 'differential_phase', 'clutter_filter_power_removed', 'spectrum_width'])
*Thread Reply:* There were two updates to NEXRAD if I remember (one was resolution update, and another was dual-pol). This could the pre-resolution update@Dan Sheldon ?
*Thread Reply:* Are the detections directly around the sensor artifacts? I am often seeing that pattern of heavy detections in a ring then dropping of, what's going on there?
*Thread Reply:* The radar beam points upwards at an angle — so most detections are near the radar as the migration is most intense at lower elevations.
*Thread Reply:* thanks, so they should be seen as genuine detections, and the detection probability drops off. Rather than artifacts that should be filtered.
*Thread Reply:* There isn't a MistNet for python? If i do get this grant, I'd be interested in porting it to pytorch. I maintain a few pytorch ecological related models -> https://deepforest.readthedocs.io/
*Thread Reply:* There is a mistnet python (not sure it’s public facing yet)
*Thread Reply:* no rush, this is not at all pressing, just folding prelim data into grants.
*Thread Reply:* Btw, Dan and I would be happy to chat sometime. Good luck with the proposal (end of semester here, so rather busy with all that)
*Thread Reply:* thanks, i'll leave a gif I just made here (wow matplotlib takes awhile to make animations). Thresholded reflectance above 0, cross correlation below 0.75. Definitely some action in there. Too early to say what happens, but some massive explosions around midday, that end quickly (unlikely to be weather?). Black cross are known historic bird colonies (not active every year).
*Thread Reply:* Exciting! Big questions are the extent to which the target species show up on radar, how they appear, and the extent to which they can be separated from other sources of reflectivity, including weather and also other species. This would depend a lot on both bird behavior and radar characteristics • Behavior: how concentrated are they while roosting, how synchronized/organized are their flights, especially how high do they fly. • Radar characteristics: mostly how far are the colonies from radar, which determines how high the beam is. In your animation I see a big exodus from ~10:40–11:30 UTC (6:40–7:30 EDT), which is right around local sunrise (7:11 EDT). This looks very much like rather widespread activity of birds taking off from those areas right around sunrise. The areas are highly correlated with your colony markers but the reflectivity is widespread, i.e., coming pretty evenly from the whole surrounding area and not closely concentrated around the markers. I would guess it could include lots of different species, e.g., waterfowl, in addition to the waders. From this example it looks very encouraging that you could measure “morning bird activity in the Everglades” but depending on the mix of species and their relative abundance it’s unclear the extent to which you could measure a specific group of species. (This is only one example, of course!)
This is from 2021, so would have dual-pol data. The dual-pol upgrade was in 2012-2013.
Resolution is 250m x 0.5 degrees. The radar is sensitive enough to detect one large-bodied bird in a volume that size, but the usual situation is many birds in one volume.
Happy to chat some time.
*Thread Reply:* Definitely, thanks for your time. I'm using these filters, i'll post one more when its done. Very slow animations building in matplotlib. I had thought it was mosquitos or non-filtered weather. I'm going to wrap this into some functions and create a sample graph throughout the year to get a sense of the 'background bird activity' in the everglades and see if we see any additional boom as the birds come in.
gate_filter.exclude_below("reflectivity",5)
gate_filter.exclude_above("reflectivity",30)
gate_filter.exclude_below("differential_phase",30)
gate_filter.exclude_above("cross_correlation_ratio",0.8)
based on figure from https://esajournals.onlinelibrary.wiley.com/doi/10.1002/ecs2.1539
*Thread Reply:* By the way if you just want to view the data, NOAA’s weather and climate toolkit is the easiest way. It’s a bit arcane but has good functionality, allows you to browse the inventory, select files, make animations, etc.: https://www.ncdc.noaa.gov/wct/install.php.
*Thread Reply:* For thresholding with dual-pol, people usually just use correlation coefficient < 0.95 = biology, otherwise weather.
A well-known figure here, @Sara Beery is giving a talk in our Learning Machines Seminar tomorrow Thursday, at 15:00 CET. Auto Arborist: Towards Mapping Urban Forests Across North America. The seminar is free and open to all, and you just connect using Zoom, no registration necessary! For info about upcoming seminars, there is an optional mailing list. https://www.ri.se/en/learningmachinesseminars/sara-beery-mit-urban-forest-mapping
*Thread Reply:* We also have a series of recordings of previous seminars on youtube, with speakers such as @Devis Tuia and Frederik Kratzert which are definitely worth checking out: https://www.youtube.com/watch?v=eP79uTWTTjY&list=PLqLiVcF3GKy1tuQFoDu5QKOM6S33t_4R1&index=8&t=1063s https://www.youtube.com/watch?v=Wrv-IS3wf80&list=PLqLiVcF3GKy1tuQFoDu5QKOM6S33t_4R1&index=1
*Thread Reply:* For those of you who missed this very interesting talk by @Sara Beery, the recording is on youtube! https://www.youtube.com/watch?v=Ob7XfUPmKu4&list=PLqLiVcF3GKy1tuQFoDu5QKOM6S33t_4R1&index=1&t=891s
*Thread Reply:* of someone needs information about the project (we have 13 phd projects after all!), the application process, etc, please reach out to me, @Blair Costelloe or @Tilo Burghardt !
*Thread Reply:* Are you set on elephants? I’m wondering if maybe there could be large animal conservation projects regionally closer to Umass that might make the logistics easier for you. For instance the Wolf Conservation Center in South Salem, NY, ~2hr drive from you. I have no affiliation, but just a thought of something to look into. There might be similar regionally-relevant issues to explore in regard to bobcats, coyotes, bears, etc. that conflict with human activities.
*Thread Reply:* I think elephants were where I started looking into wildlife conservation (from growing up in South India) but I'm not set on it! These are actually really great, having resources and expertise in proximity is very helpful when I'm still new.
*Thread Reply:* Hi Sankaran- I am a PhD student at UMass Amherst too! I actually work on plant monitoring using ML, but happy to chat if you ever want to brainstorm stuff with a biologist
Hi everyone, there is a new camera trapping dataset on lila.science: Lindenthal Camera Traps. This dataset was captured in a wildlife park by a stereo camera, which provides not only color / IR, but also distance information. Feel free to contact me if you have any questions about the dataset. Thanks @Dan Morris for the hosting and help!
*Thread Reply:* Awesome, congrats on the release @Timm Haucke!
*Thread Reply:* ACM COMPASS is one conference that is relevant https://compass.acm.org/. I've also seen a lot of CompSust work shared at AAAI and IJCAI venues
*Thread Reply:* Hey @Ando Shah, nice to see you! On the conservation side, high-profile work is still concentrated in Science/Nature/PNAS which all obviously have interdisciplinary readership, though there are many discipline specific journals like Conservation Letters, Conservation Biology, or OneEarth (the relatively new flagship Cell press journal in this area). Conference-wise, AGU is a fair-sized conference and probably a nice venue for such work (long a focus of geospatial informatics), though this would be well-received at conferences like ESA and narrower ones like ISEC https://imstat.org/meetings-calendar/the-6th-international-statistical-ecology-conference/.
*Thread Reply:* This sort of the opposite of what you asked about, but Methods in Ecology and Evolution has been very friendly to papers about new ML techniques for conservation data processing:
https://besjournals.onlinelibrary.wiley.com/journal/2041210x
If I had to generalize based on what I've seen, I would say journals like Methods in E&E that are interdisciplinary but "ecology-first" have a high bar for usefulness, but don't need to see algorithmic innovation; CS venues like AAAI/CVPR/etc. have an inherent high bar for innovation but are more comfortable with domain-specific case studies that aren't necessarily practical yet (i.e., "our technique could be applied to this interesting conservation problem").
*Thread Reply:* Not an answer to your question, but I love the second question especially because I am dealing with this and trying to pivot and wondering what degree to place myself in.
*Thread Reply:* Thank you all for those super helpful thoughts and insights!
Today, William Lidberg from Swedish University of Agricultural Sciences is giving a talk on AI-based forest landscape mapping using remote sensing. 15:00 CET on Zoom. https://www.ri.se/en/learningmachinesseminars/william-lidberg-slu-geographical-intelligence
*Thread Reply:* This was interesting to listen to! Thank you
*Thread Reply:* Good to hear @Sean Nachtrab! For those of you who missed this inspiring talk about an important application of state-of-the-art AI (maps of soil and wetlands using LIDAR data and machine learning), it's available on youtube! https://www.youtube.com/watch?v=Ezwgsjh2Oh8&list=PLqLiVcF3GKy1tuQFoDu5QKOM6S33t_4R1&index=1
*Thread Reply:* Welcome Vanessa! I've just started working on a project to identify jaguars and other Central American felines in video data, so I'd love to connect some time 🙂
*Thread Reply:* Hi @Josh Seltzer and @Vanessa Suessle At Wild Me we have data and benchmarks that might help for jaguars, leopards, cheetah, and snow leopards. Imagery not video. But happy to collaborate on this.
*Thread Reply:* Hey @Jason Cremerius that sounds awesome! I had been looking at the Leopard ID 2022 dataset and wondering how well that might support transfer learning to jaguars, it'd be a pleasure to collaborate 🙏
*Thread Reply:* Hi @Josh Seltzer and @Jason Holmberg (Wild Me) I would be very happy to collaborate as well. The ideas sound great. I would love to chat with you on those topics. I am in CET timezone. Feel free to suggest a time and date.
*Thread Reply:* @Vanessa Suessle hey, sorry for the delayed response! I am pretty flexible next week, I can do 9am (15:00 CET) or any time around then any day except Thursday. Let me know how that sounds, and @Jason Holmberg (Wild Me) it'd be great if you could join and/or we could fill you in after.
Cheers and have a great weekend 🙇
*Thread Reply:* I am available to meet his week or early in the new year. Let me know.
*Thread Reply:* Hello, I am so sorry. I missed the notification of the message. Suggest any time for this or next week I can make it work.
*Thread Reply:* Hello all, I would still be interessted to meet and have a chat. In the upcoming weeks I dont have much fixed meetings and could arrange to meet at almost any time.
Sorry that my first reply took so long. I am kinda new on Slack and missed notifications. Vanessa :)
*Thread Reply:* Sounds really cool! I'm taking a look at MapInWild but since I don't think it's addressed there (maybe I am mistaken), I'd be really curious to hear your thoughts on the temporal aspect of what it means to be 'wild'. For example, in Latin America there seems to be a common trend of "reforestation reversals" where reforestation efforts are only sustained for a few years, before land is again repurposed for other uses. I imagine it must be very difficult to account for these complexities.
(Not trying to poke any holes - I am new to forest ecology, so just really curious if/how these kinds of complications are considered!)
*Thread Reply:* Thanks Josh! Yes, in this paper we present our initial efforts in addressing the task of wilderness mapping by forming both sensitivity analysis and semantic segmentation setups on spatial/spectral levels. Although I have not addressed the task from that perspective yet, temporal aspect could introduce a new dimension where discriminative patterns are more noticable and provide valuable hints that are covered otherwise, I would say.
Yes, undisturbance is a vital criteria for wilderness areas in both gradients (spatial and spectral). Probably not what you want to hear but I would approach the reforestation reversal effects from the CV/ML and formalize an optimisation task (I would simply put it on the shoulder of a learner).
A side note: While doing that, I would also not create a strong connection between wilderness and forested areas (as wilderness areas can be found in all kinds of ecological/climate zones).
*Thread Reply:* Sounds intriguing! As a side note, there is a ton of literature on this from indigenous and political science perspectives, which I think can be very useful to contextualize the concept of wilderness/wildness in your work. Also, as a result of the "Wilderness Act" in the US, there are actually a lot of documents that outline quantitative metrics for managing wilderness areas here, even if the concept itself is more of a narrative goal for managers. In many ways, these metrics are viewed as loss functions - things to minimize in order to maintain "wilderness character". See below - cheers!
https://www.pnas.org/doi/epdf/10.1073/pnas.2022218118
https://wilderness.net/practitioners/toolboxes/wilderness-character/default.php
*Thread Reply:* Great sources and perspectives Ethan, thanks a bunch for your input! Exactly, wilderness areas are home to many indigenous communities and we should do our best to keep those areas undisturbed. But, as a cultural construct that changes in definition from country to country, it is not really easy to delineate those areas from space. That is why I find the studies on the quantification of wilderness management quite valuable for our investigation.
*Thread Reply:* Do you mean based on ground-level images?
*Thread Reply:* I meant on there positions, like GPS data, but yes at ground level.
*Thread Reply:* Do you mean which gps-collars? What type of animal?
*Thread Reply:* GPS have out faced VHS as the technology has gotten better and relatively cheaper. GPS collars are more convenient. There other mechanisms to track animals too, one was being developed lately, the ICARUS initiative using a specific satellite but Russia cut the chord lately so we’ll see what happens there. My advice is search “telemetry” and your organism of interest and see what have people have used in the latest literature. I think someone made a tracker with raspberry PI too, or maybe i dreamed it but so far GPS are the go-to protocol especially for mammals at least. Good luck!
*Thread Reply:* Here is a really cool paper from this year on all telemetry related to animal movement and ecology. It's a good starting point.
*Thread Reply:* I just noticed you wanted to track specifically in urban environments, when it comes to tracking devices for terrestrial animals, your main concern will be body weight and budget. Landscape wont have a big impact on your choice. The ones we used were Lotek, and they were pretty good, some stayed on for 1-2 years on urban coyotes. https://www.lotek.com/
*Thread Reply:* Thanks @Tiziana Gelmi Candusso!
*Thread Reply:* Hey @Pen-Yuan Hsing! I saw this shared on Twitter last week—I am guessing there is a lot of other work dealing specifically with species/individual identification, but it seems like there is very robust/comprehensive imaging techniques available, in case you're interested: https://www.nature.com/articles/s42003-022-04282-z
*Thread Reply:* WildMe https://www.wildme.org/ doesn't do this for butterflies yet, but I bet their pipeline could do it given some initial images to set it up. WildMe does species identification and identifies unique individuals for species like whales, turtles, amphibians, and various carnivores 🙂. You might want to look into the Hotspotter Algorithm for individual identification.
*Thread Reply:* INaturalist is pretty good for butterflies auto id in a large part of the world: https://www.inaturalist.org/
*Thread Reply:* Sorry for my late response here, thanks for the suggestions I'll take a look! I have a butterfly colleague to whom I will also share this.
*Thread Reply:* Hey! I am talking to a group at Mila who is working on this - let’s touch base soon? They are also in this slack actually!
*Thread Reply:* Hi @Katelyn Morrison cool! The colleague I know is Denise Dalbosco Dell'Aglio (part of a group at the Univ of Bristol where I happen to also be, but part of a unrelated project) who has a mountain of butterfly footage from Panama. Do you know them?
*Thread Reply:* I'd love to touch base about this!
*Thread Reply:* My background is in ecology/conservation but not butterflies specifically, but would be interested if we can make something interesting out of this.
*Thread Reply:* Ohhh awesome! I don’t know them, but yes I’d love to touch base. I am starting up a study to understand how lepidopterists collaborate with XAI to determine the ID of the butterfly. My background is computer science and human-AI collaboration. :)
*Thread Reply:* That's so cool! Lemme reach out to Denise to see if I can get a few sample clips to share.
*Thread Reply:* This sounds really cool! Welcome @Austin Greene 👋
*Thread Reply:* Thanks so much for the warm welcome!
*Thread Reply:* Hi Austin, I am interested in such custom cameras for pollinator monitoring. We wanted to order some Raspberry Pi gear and follow the guidelines of @Maximilian Sittinger, but I could't find any available on the market at the moment. I am therefore interested in alternatives / other microcomputers. Could you share some info with me about your models? How easy are the components to order, costs, put together and add custom code on them?
*Thread Reply:* You should talk to the Sentinel team at ConservationX! They’re working on ML-enabled camera traps/edge AI with Edge Impulse. Edge Impulse has also been working with the Audiomoth folks to develop edge computing for acoustic recorders. You could also check out other resources related to this through the Conservation Tech Directory (KiloCam is already in there 🙂)
*Thread Reply:* Tagging the Sentinel team: @Henrik Cox (Sentinel), @Sam Kelly
*Thread Reply:* totally forgot to do that 🤦♀️. thanks Ed!
*Thread Reply:* Thanks @Ed Miller and @Carly Batist! This looks fantastic @Austin Greene, it would be great to see how we can cross wires. I'll DM
*Thread Reply:* We have tree data and a model that would be nice to contribute. https://deepforest.readthedocs.io/, https://github.com/weecology/NeonTreeEvaluation. Plus tree species data that will be around soon. Some overlap with datasets that torchgeo @Caleb Robinson have done a great job compiling.
*Thread Reply:* Sure, let me know if I can help you upload it! The documentation is pretty good, FWIW : https://huggingface.co/docs/datasets/index
*Thread Reply:* (reviving a 2 month old thread out of absolutely nowhere) @Ben Weinstein FYI we've started putting some torchgeo stuff on HuggingFace (https://huggingface.co/torchgeo). In particular we're rehosting some datasets here, pros: • WayYyy faster download speeds than rehosting stuff on Zenodo! • We've hit a couple of cases now where one our dataset stops working because the SSL certificate of wherever it was hosted had expired. Most recently with the EuroSAT dataset https://github.com/phelber/EuroSAT/issues/10. We're betting that HuggingFace knows way more about sys admin-ing than random academics 🙂 cons: • no idea, seems good
*Thread Reply:* haha let me know if you have any questions, y'all :facewithcowboy_hat:
Hi y'all! 👋 I'm a data scientist at a small consulting company focused on social good, and wanted to share a programming competition we just posted! Competitions are generally a great way to break into the programming space and get some python creds 🐍
Today we launched an exciting computer vision challenge in partnership with NASA :female_astronaut: to identify harmful algal blooms in important water sources. Even better, it's punnily named the Tick Tick Bloom: Harmful Algal Bloom Detection Challenge!
🌊 🌿 The goal of the competition is to help public health officials detect and address algal blooms in water sources like lakes and reservoirs, which can pose serious dangers to both human and ecosystem health 📡 You can create new state-of-the-art methods using satellite imagery to protect communities across the US 💸 There's a $30,000 prize pool for the top performing teams!
You can learn more and sign up to participate by visiting the challenge here. Feel free to share with anyone else you think might be interested, and reach out if you have any questions! Happy coding 🙂
*Thread Reply:* Hey @Kateryna Nekhomiazh welcome!! I'm a UofT alum, so nice to see you here 🙇 I'm in the early stages of working on something you might be interested in, I'll reach out once it's a bit more concrete! Since you're interested in RL (and things at least tangential to policy), you might be interested in AI Economist (they also have an active slack community and lots of events) as well
*Thread Reply:* thank you so much, I will look forward to it!
*Thread Reply:* This is a great book for general ML concepts and algorithms, and it's all free - https://www.statlearning.com/
*Thread Reply:* I enjoyed working through this one. Although, it is 'deep learning' focused so there isnt any text space on other learning approaches. It has code available in the three largest training frameworks. https://d2l.ai/
*Thread Reply:* I often use the google ML crash course for that kind of thing https://developers.google.com/machine-learning/crash-course
*Thread Reply:* I personally found the presentation in Introduction to Statistical Learning quite dry. It does cover R, which ecologists like (and if you need them to learn R, then that could be important), and it does cover the theory from first principles. Despite the "introduction", it is still quite mathsy and I think you need to read quite a lot of it before you can actually use the information. I would say it's an approachable theory book to accompany more practical books. There is also a MOOC available from the authors.
Book-wise, I like Machine Learning with Applications in Scikit-Learn and Tensorflow by Aurelien Geron. It uses Tensorflow/Keras and it covers "classical" ML too. There's also the Fast AI book if you want something PyTorch focused (and I think is free?) Both are O'Reilly. I've heard good things about d2l.
Be warned that there is a lot of cruft in O'Reilly's offerings these days and I think their editorial standards aren't as high as they used to be. Manning has some nice books - I'm reading Human-In-The-Loop Machine Learning at the moment.
In terms of courses, Fast AI is enduringly popular. My only complaint is that they rely a lot on their library which I think adds unnecessary fragmentation when things like Pytorch-Lightning exist. But, the course is presented at a good level for beginners, the API is easy, and there are a load of little tips and tricks for Python/Jupyter.
For Deep Learning theory, I really think Andrej Karpathy's CS231 lectures are some of the best out there even though they're quite old (specifically the season where he taught). Also hey, it's 2022 and ResNets are still popular. He's also started a new series on YouTube which gives a fantastic overview of building a DL library from scratch. His talent is convincing you that actually the code that runs these mega networks is conceptually quite simple. The videos also have accompanying exercises in Colab, which is great. Like Fast AI he throws in a lot of neat Python stuff which you might not be familiar with (and it's interesting to see how he approaches problems and tooling). Of course there's Andrew Ng's course on ML which is the OG 🙂
*Thread Reply:* I would recommend Bishop’s book https://www.microsoft.com/en-us/research/people/cmbishop/prml-book/
*Thread Reply:* Not specifically textbooks or ML, but the Ecological Forecasting Initiative have put together a great list of educational resources and I'm sure there is intersection with ML: https://ecoforecast.org/resources/educational-resources/
*Thread Reply:* Thanks so much everyone! This is incredibly helpful 🤗🤗
*Thread Reply:* I would suggest using lecture notes and slides from a course (there’s so many, Stanford CS 229 seems to be popular though) rather than a textbook to help build a foundation in ML, and then use another resource specifically for deep learning. I learned from Kevin Murphy’s book (Machine Learning: A Probabilistic Perspective) which gave me a great foundation but is very huge and covers a lot of methods in depth - not sure is the best use of time for someone trying to get ahead to applied work.
I really liked this resource for deep learning. I wasn’t a fan of the textbook from Goodfellow, Bengio and Courville that is pretty much the standard in courses, there really isn’t another I know of that is widely adopted though. IMO the online resources and notebooks blow them out of the water, but I’ve never seen one where the sections on ML that precede the deep learning sections are very good. https://uvadlc-notebooks.readthedocs.io/en/latest/ https://d2l.ai
*Thread Reply:* A Course in Machine Learning by Hal Daumé is free to download and quite good. http://ciml.info/
*Thread Reply:* FGVC10's submission website is now open for your 4-page extended abstracts on fine-grained recognition and related topics: https://sites.google.com/view/fgvc10/submission Deadline: March 20
Seems that there will be quite some “Earth” workshops at CVPR this year! To add to the good news of FGCV10 above (and I let the cv4animals folks announce their own later 😉 ), I am also happy to announce the seventh edition of EarthVision 🌍📡! More info here: https://www.grss-ieee.org/events/earthvision-2023/ (we are still populating the website, but at least the deadlines are on…)
*Thread Reply:* details details 🙂, we want details!
*Thread Reply:* Here it is! https://www.cv4animals.com/ Please note that we accept work-in-progress papers, as we want our workshop to be an opportunity for people to interact and have feedback on their work. Hope to see many submissions from this community!
https://connectedconservation.foundation/news/satellites-for-biodiversity-award/
*Thread Reply:* cool demo! Are we looking at species predictions or ground truth? If predictions, from LiDAR only? Point density? Any publications/white paper to go with these? We work on similar themes using HSI. https://www.biorxiv.org/content/10.1101/2022.12.07.519493v1. Working on semi-supervision now.
*Thread Reply:* These are "ground truth" from the NYC 2015 tree census
*Thread Reply:* This looks super cool. It wasn’t immediately obvious if it’s open to folks from non European countries. I’m from South Africa and would love to apply. I’ll reread the details to check. Just reading the well structured themes and projects is interesting/exciting. Thanks for sharing
*Thread Reply:* It is open to all nationalities!
*Thread Reply:* The only limit is the MSCN mobility rule, dependent on the country of the hosting institution – ‘No residence or main activity (work, studies etc.) in the country of the recruiting beneficiary for more than 12 months in the 36 months before their recruitment date’.
*Thread Reply:* Awesome 😎 ah! I see that now. Thank you
*Thread Reply:* No problem! Let us know if you have any other questions as you write the application
*Thread Reply:* definitely, there is no nationality restriction, apart from not having lived recently in the country that will host you!
*Thread Reply:* Finishing my application later today! :femalestudent::skintone2::femaletechnologist::skintone2:
*Thread Reply:* Thank you for this wonderful consortium. I am excited to apply. I had a few general queries while uploading the application materials on the portal
- Project description - Does this refer to the 2 page motivation letter?
- List of publications - I already have a list described in my CV. Should I also list in a separate document and upload?
- Publication - I assumed I should upload a chapter from my thesis here, as mentioned in the requirements. Can I also just upload my entire thesis since the file size is within 10 Mb?
- References - I list out the details of my referees in a document ? Since they would be contacted in the second stage.
*Thread Reply:* Hello,
- yes, it is the motivation for you to join the consortium and for the (up to) three projects you apply for among the 13
- I guess it does not harm to upload a 2nd one
- Perfect. Otherwise try to compress it a bit or send a link to the full version if it is too big for the SDU server
- Yes, a one pager with the name and addresses of up to three references
*Thread Reply:* Noted. Thank you for all the clairifications Professor.
*Thread Reply:* I am having some trouble with the SDU portal for submitting my application… I have tried submitting multiple times today, but have not received a confirmation receipt yet. The portal says that this means that the application has not been registered properly. Is this an issue that other applicants are experiencing? Is there any other way to submit my application by tomorrow’s deadline if the portal continues having issues? Thank you!
*Thread Reply:* Looks like the application went through in the end. Apologies if multiple submissions appear!
*Thread Reply:* I would have contacted the SDU admin if the problem persisted
*Thread Reply:* @Emilio Luz-Ricca have you heard back regarding your application? I submitted as well and haven’t received anything further. Just curious. Thanks 😊
*Thread Reply:* hello Benjamin. We are going through all the application as we speak. It takes time, and especially since we are coordinating over 13 PIs. You should hear back from us in a week or two. Thanks for your patience.
*Thread Reply:* Awesome, I fully understand. And thank you 😊 I was just curious not really expecting anything so soon. Thanks for the swift response.
*Thread Reply:* https://wilddrone.eu/kick-off-meeting/
*Thread Reply:* @benjamin de charmoy I have been curiously waiting too. This link may help follow the happenings at Wilddrone
*Thread Reply:* @benjamin de charmoy @Emilio Luz-Ricca Did you receive any responses so far, apart from the assignment of review committee maybe ?
*Thread Reply:* I did. I was unsuccessful 🦑 Got a nice kind message 😊 keen to see the results some day as it looks like a strong initiative 🚀
*Thread Reply:* Oh! That’s unfortunate. Thanks for the reply. I am yet to hear the decision. I wish you good luck with other opportunities!
Se this post for more info and a cool video about our project: https://www.linkedin.com/posts/tsuijtenai-enabled-camera-traps-for-protecting-elephants-activity-7016335238213607424-B0e6?utmsource=share&utmmedium=memberdesktop|https://www.linkedin.com/posts/tsuijtenai-enabled-camera-traps-for-protecting-elepha[…]238213607424-B0e6?utmsource=share&utmmedium=memberdesktop
*Thread Reply:* Very cool. We are hoping to do something similar with the BearID Project as some point. I sent you a connection request on LinkedIn!
*Thread Reply:* Read your paper. Awesome work with the use of a mini computer. I am curious what will be the cost of a single compute unit?
*Thread Reply:* Super cool work! Sorry haven't read the full paper yet, but what's the power consumption like for the AI and satellite connection?
*Thread Reply:* @Jorrit van Gils @Thijs super interested in these figures. Cost and power consumption. I have understanding of AI models and deploying them. But I don't know a lot about embedded compute. Will be great if you can shed light on it or point to some relevant resources (if possible) 🙂
*Thread Reply:* @Aakash Gupta there are many technical details in the supplementary materials of the paper.
Very exciting new faculty position(s) in conservation science at Berkeley 🙂
"Please help us spread the word about our exciting new search for one or possibly more Assistant Professors in Conservation Science in the Department of Environmental Science, Policy and Management at UC Berkeley.
We view conservation science as interdisciplinary and applied by necessity; combining aspects of biology, ecology, geography, data science and environmental policy, management, and justice to dissect threats and generate solutions related to biodiversity loss, climate change, land conversion, unsustainable consumption, water scarcity, wildfire, disease, invasive species, and other challenges. We seek applicants whose work engages conservation using (but not limited to) approaches from population or community ecology, wildlife, fisheries or habitat conservation, forest or rangeland science, computer/data science or ecoinformatics, protected area or working lands management, and conservation governance, planning, policy or effectiveness.
We appreciate any help you can provide in getting this advertisement out to your groups and professional networks.
The full position description is available here: https://aprecruit.berkeley.edu/JPF03613
The deadline for applications is February 2, 2023."
*Thread Reply:* I might be able to help, depending on what you need
*Thread Reply:* Nice, but maybe not the best choice of name. There is already an existing Smalify method from Benjiamin Biggs that is quite popular, maybe you can rename with SMALifymv for multiview? https://github.com/benjiebob/SMALify
*Thread Reply:* Thanks! yes, good idea! too many similar names around 🙂
Hi friends,
FruitPunch AI 🍉 has teamed up with researchers from Cornell & Stanford, and tech partners like AWS, Edge Impulse and RFCx to work on acoustic monitoring of elephant rumbles. using AI. Up to 50 AI enthusiasts and experts from all over the world will work together for 10 weeks to improve an audio processing pipeline, optimize ML models and implement them on-edge, starting in a month from now.
With the AI for Forest Elephants Challenge, we'll develop a prototype that can be deployed in the field for follow-up research into elephant populations and to prevent poaching and human-wildlife conflict.
We have some spots left for anyone with experience in Python and about a day per week to spare between February 17th and May 2nd. Registrations are open until February 17th. I'd love to see some of you there 🙂
*Thread Reply:* This is in line with the sort of work that we do at CBI- providing science, analyses, and tools to support decision-makers. Unfortunately the fun RS + AI/ML projects happen (or are funded) less often, but we've got a couple that are active for the upcoming year or two. • Our current big project spinning up right now is for the USDA Conservation Reserve Program. The RS + AI/ML is just one component of many for this project. • We'll be doing some updates for habitat models supporting the Stephens' kangaroo rat Rangewide Conservation Plan
*Thread Reply:* I built a drone-based system for thermal rhino detection a few years ago - basically the same thing. You will probably get good results provided you have decent contrast between the ground and the animal, especially if you can do it by eye. Here's a paper from my old group that discusses thermal animal detection at a high level:
https://www.astro.ljmu.ac.uk/~aricburk/Burke2018.pdf
Otherwise you can fine-tune normal "colour" object detection models and they mostly just work. I would recommend trying to classify radiometric images rather than the min-max scaled images, there is very little published research on this, but it should give you an improvement. Might require some custom dataloading because you have to load the 16-bit images and then normalise them.
I also have labelled data from a few flights of various ungulate species in the North of the UK which might be useful, not under forest cover though.
I think the most important thing to consider is your flight altitude vs size of the animal in pixels in the image. Try and make sure that your deer are at least 30 px. That can be tricky if you're flying over forest and especially with a thermal cam because you only get 640x512 at best. Otherwise it really depends if there are only deer in the images (e.g. can you just look for bright stuff and then confirm later).
Feel free to DM me., I'm happy to have a call if you want to discuss further!
*Thread Reply:* Hi Josh - thanks for your reply! I am actually already aware of the paper you sent and have referred to it in the past, especially the observing strategy optimization web tool!
For the "colour" object-detection models, do you have any software or algorithms in mind that we should look to? I am rather new to the machine learning world and the only software I personally have experience with is ENVI and their object-oriented classification / feature extraction tool. The tool was able to pick out the deer signatures in one of our higher contrast images, though I am not sure if this is the best tool for the job.
To touch on some of the things you mentioned: due to an unknown error, all our images appear to have turned out non-radiometric despite having a radiometric camera, and yes, we are lucky that we only had deer in our survey area (or rather, they are the only larger-sized mammal on the reserve).
*Thread Reply:* Back in the day we used YOLO (for speed), but I think any object detector would be fine (e.g. Faster-RCNN, SSD, YOLOv8). There are quite a few "no code" solutions if you just want to upload a dataset and get a model out. It's more dependent on your dataset than the model architecture. I also think 90+% of people use non-radiometric, so that shouldn't be a problem even if it isn't optimal 🙂
What do you want to do with the detections? That may have some impact on which tool you choose, e.g. if you want to integrate with ENVI this is probably as good as any: https://www.l3harrisgeospatial.com/docs/ENVIDeepLearningTutorialObjectDetection.html
*Thread Reply:* This is very interesting! Thanks for sharing. For this particular use-case you should think about how the model will generalize for out-of-sample images. Does the data points allow you to identify deer vs non-deer objects in the image?
*Thread Reply:* @Josh Veitch-Michaelis Thanks for the suggestions! I will look into YOLO and the others you mentioned. We essentially have ~550 non-radiometric RGB images (like the one I attached) to iterate through, each having different levels of thermal contrast due to environmental and other factors. So, I assume we will have to take into consideration that the deer signatures will have different RGB values.
*Thread Reply:* @Aakash Gupta Thanks for your comment! Please forgive my lack of expertise in this domain, but could you specify what you mean by the model generalizing out-of-sample images? (What is out-of-sample?).
*Thread Reply:* @Savannah Bissegger O'Connor out-of-sample refers to how well your training data represents new, unseen, data you're likely to run the model on. A typical example would be you train the model in one forest, but deploy it in another and for some reason all the deer are a few degrees warmer and the model struggles to detect them.
Essentially as long as your training data looks sufficiently similar to where you'll deploy the model, you should be OK. But if you get poor performance when you deploy it, it's something to look out for.
*Thread Reply:* @Savannah Bissegger O'Connor , I believe Josh has explained it very well. To solve it you can use a technique known as data augmentation. You typically use image manipulation techniques to create variations of the image. This increases the total size of your training dataset as well as generalizing the model. But you should be vary of data leakage, while implementing it.
@here if you have 5m and you care about climate change, can you please help classify these pictures? https://forms.gle/H54vppXauJLNNMut5. Thank you! 🍀
*Thread Reply:* what's the purpose of this?
*Thread Reply:* done - very fun and took less than 5min @Sasha Luccioni aruna is a kick-ass grad student in climate misinformation at MIT. i'm assuming this is for some of her research.
*Thread Reply:* Sasha, yes, it's for a climate misinformation project. 🙂
*Thread Reply:* Thanks everyone! ❤️ all good here, just closed the form. 🙂
*Thread Reply:* spent a decent amount of time on one this summer - overall takeaways here so others can see but happy to chat more as well! • Strengths: great at everything it's stated to be good at (fast, efficient, relatively easy to use if you can get a model that is compatible) • Ease of use: have to spend some time wrangling models to get them to compile to the format the corals accept (don't remember the exact format, but there were quite a few operations used in many models that were not supported). Best-case is you're using a model that already has a well-demonstrated ability to compile to the coral format (e.g. YOLO) • Model size: must use very small models. I'd have to look up docs to remember how small this is but I think it was somewhere in the range of yolo-small or yolo-tiny. Definitely saw some performance loss due to needing very small networks, but maybe could have clawed back some performance using clever tricks elsewhere.
*Thread Reply:* @Sam Kelly @Henrik Cox (Sentinel) Did you use Google Coral?
*Thread Reply:* Hi Suzanne - Yes, we use the Coral SoM in the Sentinel. Would be happy to chat and share any learnings! Overall it’s pretty good, just some little quirks (I concur with what Alan mentioned) that I would be willing to chat about, definitely would love to hear more about how you are planning on using it. I know that @Thijs uses the Coral in some of their work.
*Thread Reply:* Thank you all for your responses! Cool, right, I was chatting with a group that is trying to use it as an end-to-end system to monitor animals in an urban setting. From what everyone has said so far, it seems superb if you can use their out-of-the-box models and pretty good if you can use small models that can work via tflite.
*Thread Reply:* Please let us know how it works for you @Suzanne Stathatos or if we can help in any way!
*Thread Reply:* I'd love to hear some recommendations as well! The Mongabay podcast sometimes covers things related to conservation tech, but outside of that i haven't been able to find many good ones. There's a ton of great conservation and climate tech podcasts i've been listening to (happy to recommend if anyone's interested) but conservation tech seems definitely underrepresented!
*Thread Reply:* I really enjoyed @Roland Kays’ Wild Animals: https://open.spotify.com/show/1M5daW5fcOtXOjYkHkrZQF
*Thread Reply:* thanks - about to release season 3!
*Thread Reply:* Also working on a new you tube channel, 2 episodes up so far https://www.youtube.com/channel/UCTMtNawoWB8-7M8waKnuiqg
@here Thank you so much for participating in my last survey. Here's another survey that should take <5m on classifying images. Some of the images in this survey can be triggering so please participate only if you are able to. Thanks in advance: https://forms.gle/xyjtGPnU6b7eygvc8. Please direct any questions to me via DM (as opposed to on this channel/on the thread) so as to not bias others about the survey.
*Thread Reply:* Some folks I know who are using MegaDetector v5 (which is just a trained YOLOv5) in the cloud, though AFAIK none via AppEngine: @Nicholas Osner, @Nathaniel Rindlaub, @Matt Hron
*Thread Reply:* Awesome, thanks for the references. I'm trying to figure out what the most "lightweight" way is to load and run the model. For "native YOLOv5" it seems I have to include a ton of libraries.
I also exported my model to tflite but I cannot really find examples on how to load and run that model (my default tflite scripts don't seem to work).
I'm also experimenting with openvino
*Thread Reply:* How are you measuring "lightweight"? If you measure by the disk footprint of your dependencies and/or the install time of your dependencies, yes, YOLOv5's native inference path is heavy, mostly because PyTorch itself is very large. But if you define "lightweight" by engineering complexity, I would consider eating the disk footprint of YOLOv5's dependency list. Their dependency setup has been tested in lots of environments, plus if you export to other formats, you likely won't get exactly the same results, which may complicate your debugging.
But if you are in an environment where disk space and/or install time are constrained, or where you already have lots of dependencies that are already compatible with one of the other export formats (e.g. TFLite), those are IMO great reasons not to use the native inference path.
Curious what you end up doing!
*Thread Reply:* Yeah, currently my app won't even deploy because of the huge list of dependencies...
OpenVino seems to be nicer / quicker and less bulky. But I find it very hard to find the correct python script to run the model with OpenVino.
@Dan Morris you mean, try to peel away as many dependencies as I can and try to stick with pytorch?
*Thread Reply:* Sometimes I really wonder how people het sh**t done in this space 😅 I've been a software engineer for over 20 years, but the whole ML space seems to be the most frustrating so far 😇 Especially if you don't "just" wanna train a model, but actually want to use it somewhere.
*Thread Reply:* It's hard for me to weigh in on whether you should try to eliminate individual dependencies or just make it work as-is in your deployment environment. Whatever issues you encountered in deployment, it may be easier to fix them and keep the YOLOv5 inference environment totally intact, as opposed to modifying the inference environment to fit your application. But, that's easy for me to say, since I don't know what those issues are and I don't have to fix them. 🙂
My simplified mental model is that all application environments are just Linux VMs that support arbitrary Docker containers, in which case it probably is easiest to keep YOLOv5 exactly as it is, dependencies and all. But that model is clearly oversimplified, and if the payloads your execution environment can run have limitations, maybe it makes sense to either prune problematic dependencies away from the YOLOv5 code or switch to another runtime.
Good luck!
*Thread Reply:* It seems like this is the best option for me: https://learnopencv.com/object-detection-using-yolov5-and-opencv-dnn-in-c-and-python/
It's exporting the model to ONNX format and then I only need the opencv-python-headless dependency to do inferencing.
*Thread Reply:* So I believe non-profits gets one free professional slack account and that would likely be the best way.
*Thread Reply:* non profits do!
*Thread Reply:* you just need to find a properly registered non profit that will give you their account
*Thread Reply:* Is the Slack for Wildlabs public? Or is that just for your internal use?
*Thread Reply:* Oh it's our internal(ish) one. It also has all the people we're collaborating with on various projects and all our community organisers/group managers in there as well
*Thread Reply:* Would we have to switch then to a new workspace (with a new name)?
*Thread Reply:* To upgrade this workspace you (the workspace owner - sara?) need to be able to input a charity number and say it's their workspace, then slack will upgrade it.
*Thread Reply:* I'm sure one of the 1374 people in this group is associated with a group that isn't making use of their free account - and so would kindly donate it to the cause
*Thread Reply:* OR - someone goes through the hassle (cost?) of registering 'Ai for conservation' as a charitable group?
*Thread Reply:* OR someone knows someone at slack that will do us a solid and just give us a free workspace. SURELY in this group you guys know people working there
*Thread Reply:* Just my 2 cents: seems like Slack free upgrade for NGO, etc. is limited to 250 users only. For more than that, there is an 85% discount -- so still some costs. the last option @Stephanie O'Donnell floated sounds best at the moment 🙂
*Thread Reply:* ooooh i didn't realise there was an upper limit. Also Andrew called this idea first!
*Thread Reply:* There's a limit, I've looked into this but it would still be $$$ for a community this size 😞
*Thread Reply:* @Ben Weinstein
*Thread Reply:* Hi Matt, This sounds exciting. I don’t have the data you’re looking for, but I’m curious, what do you mean by very high resolution?
*Thread Reply:* In case you do have bounding box annotations for each tree, you could use something like boxinstseg to predict the segmentation labels
*Thread Reply:* Re: resolution - at the moment my thinking is it would probably need to be ~30cm (I think that's the lowest resolution that it's currently possible to get? Someone feel free to correct me if I'm wrong though) to separate crowns at the individual level in areas where the species distribution is pretty homogeneous (in very diverse canopies it's quite easy to separate trees using hyperspectral data or stretching the RGB bands when you're labelling). Maybe 50cm ish would work.
*Thread Reply:* On boxintseg etc - For the purpose of assembling the dataset I'm really looking for ground truth data rather than unsupervised predictions. It would be very interesting to see whether weakly supervised approaches like this could reproduce the manual/laser scanning derived labels. I'd need the ground truth labels to check how well it was doing though
*Thread Reply:* @Matt Allen, what's the goal? Check out the last couple datasets and the https://google.github.io/auto-arborist/ https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1009180.
*Thread Reply:* I think to have a new dataset there should be some kind of new/different goal. Do you have field based ground validation? You can try coarsening the 10cm data we have in that benchmark to get a sense for accuracy. I'm not sure individual tree segmentation can be rigorously done at 30/50cm. Really really hard.
*Thread Reply:* We strongly suspect that the 80% accuracy of deepforest (https://deepforest.readthedocs.io/en/latest/landing.html) at 10cm is limited by human annotators inability to do better than that. We have cross validated across many observers, included people in the field with tablets (https://peerj.com/preprints/27182.pdf). We are really unsure if we can do better with hand labeling.
*Thread Reply:* There are hundreds (hundreds) of papers on this, many include zenodo links. Some big ones are going to get integrated here when I have a second. https://github.com/weecology/DeepForest/issues/340
<a href="https://github.com/weecology/DeepForest">weecology/DeepForest</a>
*Thread Reply:* Also check out large work being done https://openforestobservatory.org/ by Derek Young. I spoke with him a couple weeks ago and he was interested in contributing data.
*Thread Reply:* Thanks @Ben Weinstein we'll look into all of those ☺️
*Thread Reply:* Happy to zoom whenever if I can be helpful.
*Thread Reply:* I also know that the world resource institute has been doing a ton of this work for labeling/validation. By coincidence we are talking this week about data through https://www.globalforestwatch.org/. I'll know more next week.
*Thread Reply:* Thanks for all this, appreciate it - apologies for being slow to reply; it's getting (somewhat) late here 😅
*Thread Reply:* We [Restor/ETH] have around 5k 2kx2k images @ 10cm with instance segmentation labels, and we're currently field trialling. Images are from global drone data, so we generalise pretty well. Hopefully this will also be published sometime in the near future. Validation is by far the hardest problem, so we're working with partners to get ground truth where we can (it's still incredibly hard to do at the tree level, beyond asking people to eyeball images to check that the model is correct).
We opted to not segment closed canopy, because we're sceptical that you can get accurate instance segmentation with RGB alone for exactly the reason that Ben mentioned. We basically treat it as a crowd class and have the model try and distinguish between 1 vs many (and you can maybe use a different downstream task to segment the closed areas).
The difficulty with relying on other data sources that would help you disentangle is that they're hard to get consistently from restoration practitioners. LIDAR is pretty rare, though there is some evidence you can do OK with crown extraction from photogrammetry. Ideally we're targeting something that will work reasonably well on high-resolution satellite imagery which is mainly RGB (+ maybe NIR).
Would also be great to join a Zoom, I'd be happy to share where we're at.
*Thread Reply:* Sounds great, let me know when they are available and we can add them. Our model was trained only on US data and it gets applied broadly, we have no way of knowing its performance at a global scale. If you can try the release model, that would always be of interest. One very strange caveat is that we have yet to find a LiDAR + RGB integration that outperforms RGB alone. NEON's LiDAR cloud is sparse (5 points /m), which makes sense for the huge area it covers, often more than 4 million trees. Again, given lack of field validation, hard to know if because we annotated in RGB, we do better in RGB. We often see papers that use very dense LiDAR coverage and get good results, but they don't really reflect large scale field usage. This was what designers at OpenForestObservatory and I were discussing. We had a one off paper https://ieeexplore.ieee.org/abstract/document/9387530 that looked at post-hoc HSI splitting of RGB crowns. We have always meant to get back to it, but spent too long focusing on species ID.
*Thread Reply:* Yes, I've spoken to other folks and they've echoed the LIDAR bit. Weird. Not something we've tried though and our images were sourced from Open Aerial Map who unfortunately don't provide surface models.
Currently I'm looking at options to QA our labels (paid-for and automated). We're also looking at crowdsourcing for label verification, something along the lines of "place a keypoint in the centre of every tree in the image". Our masks are already pretty good, but we have a lot of annotator confusion between single/mutiple trees. I think there is also promise in sub-classing the labels to try and force the models to learn better distinctions like pine/leafless/broadleaf.
It'd be great to have a chat about benchmarking (both our data on deepforest and vice versa). We do have a fair number of orthomosaics+DSMs from partner sites so one idea would just be to look at agreement between RGB model predictions and 3D model predictions. There are also a few datasets that can be used for tree count verification e.g. there's some post-disaster imagery over Tonga that has individual trees labelled (including species).
*Thread Reply:* @Ben Weinstein I was speaking to Justdiggit today and the LIDAR question came up. Do you know what sort of pre-processing people have tried to integrate either LIDAR returns or DSMs?
*Thread Reply:* Thanks @Ben Weinstein @Josh Veitch-Michaelis we do have some TLS-segmented data in a few ecosystems and are interested in this issue of accurately identifying where there is ground verification. @Matt Allen has been looking at other methods of ground verification too (e.g. using trunk locations from DGPS/total stations)
*Thread Reply:* On the issue of the annotations being verifiable - I think the degree to which manual labels etc. are acceptable really depends on the ecosystem - for a lot of plantation type forests I think hand annotations should be fine. On ALS, I should clarify that I really meant drone-based ALS (ULS might be the right acronym) where the point density is much higher than say from scanners mounted to planes. In very dense canopy the labels should probably be derived from TLS, unless they can be verified in some other way. The aim is to see whether sensors derived from these labels can be reproduced using the RGB data alone. I don't think I would reasonably expect it to be spot on but it would be interesting quantify exactly how far off they are (similarly, could also see how close human annotators tend to be)
*Thread Reply:* One thought we had was that if we can only delineate the denser areas (whilst also identifying standalone trees), then that's still a good step towards handing off to another algorithm/method. Simple/classical CV methods can work quite well in certain environments, but are very sensitive to background inclusion (e.g. if you give an area that has no trees in, they'll happily try and predict). If you can also constrain where to run a LIDAR/photogrammtry/DSM-based algorithm they should have fewer false positives.
I'm hoping that with some crowdsourcing we might be able to better quantify human annotator disagreement in different scenes (different biomes, urban vs natural, etc).
*Thread Reply:* Its not terribly expensive - you could start an ngo for AI for Conservation. It would cost a couple hundred bucks a year, but would require a little bit more overhead on your part.
*Thread Reply:* Well, I've been thinking about it already. If there are folks with experience/cycles to help with this maybe reply in the chat and we can look into it?
*Thread Reply:* Ive never done it for an ngo, but I've started a few LLCs throughout the years. Happy to chat about it and see if I can help.
*Thread Reply:* @Aamir Ahmad expressed interest as well. I can ask Priya Donti, they just went through this process for CCAI. And maybe I can get help from MIT lawyers once I start in the fall.
*Thread Reply:* There are other opensource/academic communities which use Discord instead of slack (I’m in the NeRF studio discord https://docs.nerf.studio/en/latest/ for example). Discord is very much more orientated towards social communities rather than professional (so has a weird look and lots of things about emojis and gifs), but the past messages are free. Of course given the number of people already in the slack instance I’m sure people would rather not migrate!
*Thread Reply:* How much would the fee be? Could we find some sponsorship?
*Thread Reply:* Pro: $7.25 USD per person/month, when billed yearly $8.75 USD per person/month when billed monthly 😲
*Thread Reply:* Thanks, that is a lot indeed!
*Thread Reply:* For a group of our size, we might need Business. With 85% discount, in € I calculate about 2300€/month for our ~ 1300 strong group. Plus, our group keeps growing!
*Thread Reply:* But it might be worth contacting the Slack sales team to get a fair idea regarding pricing? @Sara Beery as the creator of the workspace, perhaps you could drop in a query?
*Thread Reply:* No sure if this might apply https://slack.com/help/articles/206646877-Apply-for-the-Slack-for-Education-discount
*Thread Reply:* Indeed, this is the discount I relied on 🙂
*Thread Reply:* For what it's worth, I've had success with Element (which runs on the Matrix network) for collaboration within a 3-year big EU-wide research project, and use it extensively in various other communities.
Uniquely, it's entire tech stack is 100% open source which is more socially responsible, and it has seen adoption by the French and German governments for internal communications, so it's not just another random app.
The free-of-charge personal tier is sufficient for our needs, and because it is fully open source there is always to option to self-host.
I suspect most would prefer to remain on Slack so as to not create another account. But for cases where we start a new community elsewhere, IMO this is much better - ethically and socially speaking - than Discord.
*Thread Reply:* @Sara Beery I just hired someone for OVAI who has a degree in non-profit administration, and may be similarly having to set up a non-profit. I'll know more in two weeks and happy to compare notes
*Thread Reply:* Hey everyone! Bourhan here, CEO of the non-profit Rainforest Connection (rfcx.org), I think the process of incorporating a company and filing for the 501c3 status is quite extensive and lengthy especially if the primary goal is to get a subscription discount, etc. The IRS sometimes takes upwards of a year to approve a non-profit status, at times even longer.
*Thread Reply:* Couple of things I can help with. We currently have a free slack subscription for non-profits and our organization was selected this year to be part of the Salesforce accelerator. Salesforce (owners of Slack) offers a 2 year free subscription as part of the accelerator for us. Given that we already have the non-profit edition, we wouldn't have any use for it so we could possibly give that up for this workspace. The key would be to convince Salesforce to do that which I don't necessarily see any issues with, though we would definitely have to make the case to them.
*Thread Reply:* Let me know if y'all are interested in that and we can talk about the next steps.
*Thread Reply:* Definitely interested! I'm happy to help make that case
*Thread Reply:* Awesome! I'll talk to our team at Salesforce about it. If you have any small write up on the purpose of this workspace etc... it would be super helpful
*Thread Reply:* correct me if i am wrong, but isnt this the third (topic-relevant!) conference in kigali this year?? rwanda for the win! :flag_rw:
*Thread Reply:* @Cody Kupferschmidt
*Thread Reply:* I think a lot of them are in conjunction with ICLR!
*Thread Reply:* yeah the proximity in date of this one would seem to be that. i think ICCB was in june or july. kinda cool coincidence i guess. maybe a long holiday in rwanda is in order? 😏
*Thread Reply:* We're organising a movement ecology workshop at ICCB in July which will include an ML focus. We also will be organising an acoustic one with ML elements at some point this year. Will share details about both when available, and they'll also be posted in the conservation tech event calendar on wildlabs - https://wildlabs.net/events
*Thread Reply:* Hi Dan! Great to see you here 🙂 I'll definitely be at CVPR and ESA, and possibly ICCV. I'm considering attending ICLR as well.
*Thread Reply:* I’ll be at ICCB! And wildlife society (Nov). Potentially IBAC
*Thread Reply:* Definitely check out the #upcoming_events channel. Also +1 for ESA and the wonders of the Pacific Northwest.
*Thread Reply:* Damn…. do I need to get myself to Portland too??
*Thread Reply:* (self-advertising here 🙂) -- We're creating torchgeo, a "PyTorch domain library, similar to torchvision, providing datasets, samplers, transforms, and pre-trained models specific to geospatial data." to make it easy for non-domain experts to use geospatial data (mainly satellite and aerial imagery) / benchmark their cool models on geospatial datasets. An awesome first contribution for anyone is to add a dataset! • Here is the docs page to see which datasets we already have -- https://torchgeo.readthedocs.io/en/latest/api/datasets.html • Here's a list of datasets that we've put together that would be awesome to add to torchgeo -- https://docs.google.com/spreadsheets/d/1TU1T5RdVWBify6MZGVVJICOX3c7qkhfmSGN2luyFNVU/edit?usp=sharing Also relevant, here is our "Contributing" page (https://torchgeo.readthedocs.io/en/latest/user/contributing.html) that attempts to walk potential contributors through all the extra complicated looking stuff that happens in "Github Actions".
We (5 maintainers) are pretty active on github so feel free to open an Issue if you want to discuss the details of a dataset before or while you are implementing.
*Thread Reply:* Thanks for this thread! I hadn't seen the long list of projects at opensustain.tech
But I have seen a number of ecological or domain-specific projects that could certainly use help by "general" python, database, devops and front-end engineers! Here are some ways that come to mind or that I have helped projects accomplish:
• Converting code in notebooks to python modules • Finding bottlenecks: writing faster database queries, adding caching layers, turning one-at-a-time steps into batch functions. • Containerizing code (docker), making the app portable to multiple environments, making it easier for more developers to on-board & contribute • Adding support for another platform (windows, linux, or perhaps R to python) • Writing tests, implementing automation, CI/CD, making things reproducible in general • Writing and improving documentation • Utilize packages, patterns and 3rd party integrations that are common in software world, but not in the science world (black, pep8 formatting, Sentry, NewRelic, GitHub Actions, AWS S3 open license storage, etc) • Updating dependencies and testing with those versions • Improving the management of data in general, perhaps switching to a database, or making it easier to share and retrieve large files or images. Removing large files & log files from the git repo. • Improving or creating a UI • Make the project easier to configure via environment variables, CLI parameters, etc. • Adding python type annotations • Changing hard-coded local absolute paths to relative ones 🙂 • Removing sensitive information from the repo & git history
*Thread Reply:* Those all sound like optimal ways to get new engineers involved in the conservation space. Do you have pointers to specific projects?
*Thread Reply:* The AI for Wildlife Lab at FruitPunch hosts 10-week Challenges accessible to anyone with some basic programming skills. There's domain experts involved, but usually in a stakeholder role and to give masterclasses to the participants. Most of the work is done by ML/DS enthusiasts with an interest in conservation, rather than conservationists with an interest in ML/DS.
Enrollment for an acoustic monitoring Challenge is open until Feb 17th. (AI for Forest Elephants) Usually a new Challenge in conservation launches every few months.
*Thread Reply:* Please also check Omedna challenges: https://omdena.com/projects/ There are projects and local chapters which you can join. Some interesting ongoing projects are related to extreme weather forecasts, monitoring air quality and preventing wild-fires
*Thread Reply:* @Michael Bunsen ❤️ what a great list. I'm saving this for future reference. Do you have links to any specific examples you can share?
I'm particularly keen myself to become more familiar with streamlined / most used ways to do these things in python. having been in the R world a long time I'm familiar with which are better/more popular patterns for caching, unit tests, containers, dependency management, linting, documentation etc, but trying to cross over into python I rarely know where to start!
*Thread Reply:* One more note to add here... I mentioned opensustain.tech in my original post; I noticed that they also have a really useful table where they index a lot of GitHub metadata, including whether each repo uses the "good first issue" tag:
https://airtable.com/shr9we419r2TkpLkc/tblfcyw3opQsmaqQj/viwtjrGUtJZG6yGBH?blocks=hide
Sorting that table in descending order by "good first issue" seems like an awesome way to find projects. Just the fact that a repo uses the "good first issue" tag at all is a decent proxy for "we can help on-board new devs".
Also FWIW there are not a ton of wildlife conservation projects there, so if folks here have OSS projects that might be interested in external contributors, add your projects!
*Thread Reply:* Here is one Python package for estimating how many beavers a given stream can support. I got excited about after it was mentioned in a published book about beavers ("Eager"). However when I went to go check it out I found it was impossible to run. Multiple file paths are hard-coded to a person's local workstation and it must be run within ArcGIS (also Windows only). It would be awesome for someone with knowledge about spatial data & python to re-work this package to run outside of arcgis (using shapely, etc). Or perhaps just port it to QGIS. https://github.com/Riverscapes/pyBRAT/
For Java developers, or big data engineers, it would be awesome to give GBIF some love! It's a big repository for biodiversity data from many sources (including iNaturalist). Any improvements to GBIF will help out many conservation efforts around the world. https://github.com/gbif
Catalog of Life & Checklist Bank are also big projects that need help. These are working hard to create harmonized taxonomies for all walks of life. As well as create tools for different organizations to merge and keep up with changing taxonomies. They have a newer javascript/html portal that needs help. https://github.com/CatalogueOfLife
*Thread Reply:* One more! I think Android / iOS apps for DIY camera traps would be very helpful for many projects. Here is one I know if now that could use some help "BioLens" (maybe formerly "AutoMoth"). Perhaps someone can add support for Tensorflow Lite models?? https://github.com/bhostetler18/BioLens
*Thread Reply:* Hello! I was wondering if this was recorded? I would love to listen and learn, but just missed the seminar. Thanks!
*Thread Reply:* just voted, good luck with the grant application!
Hello everyone, we setup a special issue to take some time to thing more broadly about the pro and cons of automation in conservation and ecology https://www.frontiersin.org/research-topics/53052/can-technology-save-biodiversity . Feel free to propose a contribution! Thanks!
*Thread Reply:* Oh nice idea! 🥳 I actually came here today with the intention of posting my calendly and asking if anyone wants to chat and meet 1:1, but I'll wait until after the group coffee. And maybe then if there are more people who are interested in something like this we can make a random-coffee-partner channel. 🌱 (and yes please I'd love to be added to the calendar invite kalindi.fonda@gmail.com)
*Thread Reply:* Love this idea dan! Incidentally... this just so happens to be the half hour before our Feb Variety Hour, so you can roll into an all our conservation tech gathering if you get excited! https://wildlabs.net/event/variety-hour-february
*Thread Reply:* OMG, I somehow totally missed that, even though I literally searched for one and only one thing to not schedule over, and it was the WILDLABS variety hour. For some reason I just missed this. I 100% do not want to schedule over 30 minutes of that, and two hours of community-ing is a lot, so I'm going to move this hypothetical AI4C coffee a day earlier. My bad!!!
Editing my original post to change to Tuesday 2/21. Sorry!
*Thread Reply:* Well 100% my fault as I literally only just published it
*Thread Reply:* but as it's apparently helpful - future reference, our event schedule is as follows 😂
*Thread Reply:* March 29, April 26, May 31, June 28, July 26, August 30, Sep 27, oct 25, Nov 29. Always the last wednesday of the month. AND - if anyone would like to speak about their work at any events or has someone we should feature, please let me know!
*Thread Reply:* This sounds like fun! I’d love to join: Daniel.Elias.Velasco@gmail.com
*Thread Reply:* Meeting link for tomorrow's (Tuesday 2/21) AI4C Coffee "event" @ 7:30am PT:
"Event" is in quotes because, as a reminder, I have no plan at all for this hour. This is not like a WILDLABS event where there are well-curated games and sound effects, this is just "here's a meeting link, YMMV". For the same reason, there is a distant possibility that we exceed the number of participants in a regular meeting, in which case... see above, YMMV.
That said, here's my ask to everyone who joins: when you join the call, post one sentence in the chat about what you do for work, and two sentences in the chat about what you do for fun. If there is awkward silence, I will pick someone at random and ask more about your hobbies. If it is a total cacophony, I won't try that hard to fix it, but it wouldn't hurt if others were ready to spontaneously create their own meetings for side chats, e.g. if I find someone who only wants to talk about Fender basses, I won't hesitate to talk about Fender basses for an hour, so eventually you'll all have to say "hey, Dan, take this to a side chat".
YMMV!
*Thread Reply:* Thanks everyone for joining! We learned that AI for Conservation folks like to climb, and that underwater rugby is a thing.
*Thread Reply:* Fun meeting. it was great meeting you all!
*Thread Reply:* 🌊 Oh yes underwater rugby 💙, here's the map with the teams: https://www.uwrmap.com/ let me know if you are actually interested in trying it out and I can connect you with the people, but even if you just show up, people are usually very welcoming.
*Thread Reply:* oh drat. only just now noticed this got moved ahead to today. 😭 very sorry to miss you all. i guess i will wait til next one. (at least i have consolation that i have met some of you, including dan, in person, right? wahhh) 😁
*Thread Reply:* Thanks for organizing, Dan! If anyone wants to chat about marine stuff 🐟 or hiking 🧗♀️, don’t be a stranger
New NASA Applied Remote Sensing Training Program (ARSET) relevant to this community: NASA ARSET has opened a new open, online intermediate webinar series: Biodiversity Applications for Airborne Imaging Systems. This four-part webinar series will focus on NASA Earth Observations (EO) that can be used to characterize the structure and function of ecosystems and to measure and monitor terrestrial and aquatic biodiversity.
If you have any questions, feel free to email natasha.r.johnson-griffin@nasa.gov. More info on the training and how to register for free can be found here: https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgo.nasa.gov%2F3DkCnvs&data=05%7C01%7Ccbradley%40biologicaldiversity.org%7C64efe90aaab84844613b08db0b93084b%7C95c0c3b8013c435ebeea2c762e78fae0%7C1%7C0%7C638116498368338551%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=R%2BPkvKOgpk%2FS8EswcwjlMjwVfHqn5b8tbFqPijmfhT4%3D&reserved=0
Hi everyone, our group is once again searching for one or more Research Assistants to contribute to our bioacoustics research projects. A description of the opening and how to apply can be found here - https://www.kitzeslab.org/research-assistant-position-available/. Feel free to be in touch with any questions!
Hi all. It's world pangolin day! so no better a time to share the news that the Paul Allen Family Foundation have funded a six year programme called Operation Pangolin to do something serious to address their decline, starting in West and Central Africa. The good news for this community is that it will include the use of open source conservation technology, with Arribada leading on development, but importantly will aim to develop reliable in-field data streams for AI / ML ingestion upstream (think regular inference data from camera traps, acoustic recorders etc but on a regular basis). I hope many of you can get involved to help the humble pangolin not to be turned into medicine and ornaments. More here: https://gfjc.fiu.edu/operation-pangolin/index.html
*Thread Reply:* Hi! Some useful resources to look into - the Conservation Tech Directory, WILDLABS Acoustic Monitoring Group, and I’ve got some resources/lists of review papers/etc on my website 🙂.
*Thread Reply:* Dan Stowell's paper is a good place to start
Just saw that Appsilon released a new version of Mbaza AI - https://appsilon.com/mbaza-ai-update/
*Thread Reply:* Hey @Carly Batist, it was updated in October of last year but due to [many reasons here] I never got to write that up for our blog, we shared it with the current userbase. There is a lot of exciting stuff coming to Mbaza soon**, and once that happens I'll make sure to share it with the community better. Stay tuned and so on.
*Thread Reply:* It sounds interesting to me. I work at a university in Colombia and have heard of some related efforts. Feel free to DM me if you would like to chat about it
*Thread Reply:* Yay thank you! Colombia has some amazing innovation happening in this space.
*Thread Reply:* Savimbo and KUNGFU.AI Partner to Bring AI to Rainforest Conservation
*Thread Reply:* The term is often used and abused e.g. some species tasks are not necessarily fine-grained e.g. a duck versus a pigeon.
We often show the image below at our fine-grained workshop.
There is also a huge literature in human vision related to subordinate classification that you might have seen. https://link.springer.com/article/10.1007/s004260050047
*Thread Reply:* This image is useful, thanks - I like the way "instance recognition" is included, which is perhaps the best-defined level in the hierarchy - we don't want to generalise across instances, at that level. I still find unclear the distinction between the top 2 levels. Perhaps because human visual perception is not a relevant starting point for me! 😉
*Thread Reply:* Even instance is not totally ambiguous e.g. what if I buy two copies of the exact same chair ... But agreed, it is likely the most well defined.
*Thread Reply:* I often think of it as a "how easy is it to tell these things apart" spectrum with no clear boundaries, that is somewhat individual rooted in each person's expertise/experience. For example, folks at the Lab of O find it VERY easy to tell apart bird species that I would easily lump together, and I'm pretty good at pretty niche stuff like matching ballet choreography to specific choreographers, etc. Course categories would be things that don't require any training to distinguish. But there's definitely cultural and regional differences in what those might be, which is interesting.
*Thread Reply:* curious which species you are working with re: individual id. we have quite a bit of experience with it over at wildme.org and i would definitely say it falls into "fine-grained".
*Thread Reply:* One nice attempt to formalize “granularity” in a machine learning context can be found here:
https://arxiv.org/abs/1912.10154
*Thread Reply:* Thanks all! Jon - we're working on developing individual-ID methods that work across many species - so the focus is very much on the ML methodology. Having said that, we're working with a collection of different bird datasets, as well as various terrestrial mammals (...based on sound!)
*Thread Reply:* @Peter Kulits is also digging into multi-species re-ID, maybe you two should sync up and see if the efforts are complementary?
*Thread Reply:* Thanks! Elijah - thanks for the paper. I was actually quite surprised at their way of formalising the notion: essentially how neatly-clustered are the classes. I'll take some time to think about whether it's a good measure for us. Intuitively, I'm more drawn to the directly hierarchical formulation. Thanks
*Thread Reply:* happy help, but heading to Colombia. @Howard L Frederick message me again in a week or so. https://deepforest.readthedocs.io/en/latest/bird_detector.html
*Thread Reply:* 👋 Hi @Howard L Frederick - we are working on a model to automate the counting of cormorant (shag) nests in drone imagery from Antarctica, here's a summary of a quick POC we did last year: ➡️ https://appsilon.com/yolo-counting-nests-antarctic-birds/
Now last time I checked the final model was almost ready, and we should be able to share more on this soon. A paper is also in the works, but as with those, it's coming in several months at best.
Perhaps not 1 to 1 what you are looking for but I expect you can get some good insights and I'll be happy to pass any specific questions to our ML team and see if we can help (email me at andrzej@appsilon.com, DM here, or via LinkedIn).
*Thread Reply:* That's a very good start - will be in touch directly!
*Thread Reply:* I suspect the chaotic backgrounds from African savannahs will be problematic but this is a good basis to start with
*Thread Reply:* okay, back from Colombia. Can you provide a single full size image and we can test our bird detector. It has been trained on 200,000 images, so you only need a tiny handful to customize to your situation. We tend to do annotations in QGIS. There is a video in the link above.
Hi, I’m Adam from Edge Impulse. Helping SmartParks with ElephantEdge, AI powered tracking collars, the folks from Conservation X who created Sentinel, an AI powered nature camp, and WildLabs with wildlife research grants. Here to learn and help as I can. I help these organizations on a few projects and always want to learn and do more.https://techcrunch.com/2020/11/20/can-artificial-intelligence-give-elephants-a-winning-edge/
I just want to share with you that
It's about how we are using smart cameras with machine learning to try and mitigate human-elephant-conflicts in Gabon.
Please let me know what you think of it! 🙏
*Thread Reply:* I look a bit scared in the YouTube preview image 😂
*Thread Reply:* Great stuff! What kind of model do you use for detecting the elephants/persons? And how long does it take to process one image on the embedded device?
*Thread Reply:* @Peter van Lunteren We use a TensorFlow Lite image classification model, it takes ~100ms to classify a single image.
*Thread Reply:* I once (very briefly and not at all thoroughly) played with the idea of running the MegaDetector model on a raspberry pi with a camera and motion detector attached. The idea was similar, to detect poachers or animals. One of the problems was the processing speed on the embedded device (especially for such a huge model like MD). You’d have to create something like a queue and hope for not too much false positives. Or train your own light weight model, but I didn’t really want to give up accuracy. And the other thing is that the existing commercial camera traps will always be better and than some raspberry pi with a camera attached. Anyhow, long story short… good that you found a way to deploy this! Very impressive
*Thread Reply:* Thanks Peter! I think there are some people that got the most recent version of MD running on a Pi. But I don't think this is an ideal solution.
We did use MD to preprocess out dataset and we basically trained our own model to detect elephants and humans. (Together with appsilon)
*Thread Reply:* Good stuff @Thijs 💪 and cheers for the shout out!
*Thread Reply:* Indeed, I am based in Lille, France and I work remotely for Edge Impulse 🙂
*Thread Reply:* Is it gonna be recorded? 🙂
Hi everyone, I’m with the project developer Savimbo,
We’re building a really cool #ML algorithm Measuring jungle from the ground with #smallfarmers. This is a super novel application of AI for conservation.
This project, which is only possible because of KUNGFU.AI, is super simple, and at the same time, vastly important. Smallfarmers scan the jungle with photographs, and the algorithm says whether that jungle is intact. The algorithm is intended to scale on-the-ground deployment of microeconomics that stop deforestation, but the dataset has wider applications for species identification.
.jpg)
*Thread Reply:* Hey @Jinsu Elhance, I'm currently a PhD student at Edinburgh/Leeds SENSE Centre for Doctral Training in Satellite Imagery for Environmental Applications, which may have projects you will be interested in.
While the projects are more application based, as opposed to focusing on creating general methods of super-resolution for satellite imagery, these kind of tasks are often baked into the processing work done for an ecological task e.g. improving satellite imagery to better identify land cover types
I believe the initial round of applications is closed for this year, however there are often a handful of projects available throughout the year, so if interested then this is the place to keep an eye on: https://eo-cdt.org/projects/
*Thread Reply:* I don’t know if at the group of Prof Jan van Gemert at TU Delft (Netherlands) they work on this specific topic, but they are cool guys and Jan might be interested in a new perspective. Perhaps, have a look at their publications to get an idea?
*Thread Reply:* Hi! I would also like to join if possible!
}
Migration Dept., Max Planck Inst. Animal Behavior
(https://twitter.com/animaltracking/status/1630944261347725313)
Twitter
Hi everyone! In case you want to watch a Stanford/Hopkins Marine Station seminar given by our own @Sara Beery, you can register here! I’m sure she loves it when we advertise her talks here. 😉 https://events.stanford.edu/event/sara_beery_computer_vision_for_global-scale_biodiversity_monitoring
*Thread Reply:* @Sara Beery if you’re in town, we should have you come and visit MBARI
*Thread Reply:* Sure! I'm driving up on Thursday, and was planning to drive back Saturday. We could maybe do something Thursday afternoon?
*Thread Reply:* Let me follow up over email. It’s an especially tough week for me to organize things since I’m out of doggie-care, but it would be great to connect. More soon.
*Thread Reply:* Hi @Sara Beery, Thanks for the talk on Friday, It was really interesting to hear about what you’ve been working on. I was wondering if you would be able to share a link to the talk recording and/or the slides so I could look back at some of the things you mentioned and share with others. I was also really interested in the Alaskan fisheries project you mentioned and wondered if you might have a name or link for that?
*Thread Reply:* I'll DM you my slides!
Here is the github repo for the sonar fish counting project: https://github.com/visipedia/caltech-fish-counting
Work led by @Justin Kay!!
*Thread Reply:* There's no right answer; everyone's problem is a little different, and the optimal workflow depends on the details of your problem, your team, your institution's data policy, etc. But a couple links that may be helpful...
I try to keep a list of systems that exist (with or without AI, local or on the cloud) here:
https://agentmorris.github.io/camera-trap-ml-survey/
And we can usually help narrow down your choices if you can answer the questions here:
It may be interesting to have that discussion here, but if that exceeds what's practical to type into the Slack reply box, feel free to email cameratraps@lila.science .
*Thread Reply:* Hi Tyus, interesting topic. What co-variates are you considering to characterize the human-dominated landscape that can help explain the predator’s occupancy?
*Thread Reply:* Great question, I am looking at housing density, population density, road distance, vegetation cover, and residential housing distance. I'm trying to see if I can gather noise levels but we will see.
*Thread Reply:* did you already check out TrapTagger? Your case sounds like a perfect fit
The GeoLifeCLEF23 competition, within FGVC and ImageCLEF, is also open! Submit you predictions for plant species presence/absence using satellite images and time-series before May 17th https://www.kaggle.com/competitions/geolifeclef-2023-lifeclef-2023-x-fgvc10/
👋 Hey Folks!
📣 Exciting news! We just published 🔗a new article on Mbaza AI.
Mbaza AI featured here has already established a strong presence in Gabon and is soon expected to expand to Kenya (thanks to Ol Pejeta Conservancy).
More details below 👇
🤖 Ol Pejeta folks needed an accurate model to confidently automate the wildlife identification task; they provided us with a subset of manually labeled data from 2018 with images from 9 different cameras in 2 migratory corridors.
🦏 Ol Pejeta Conservancy is over 90,000 acres in central Kenya committed to conserving biodiversity, protecting endangered species, driving economic growth and improving the lives of rural communities.
Please let me know what you think in the comments! 😄
If you have any questions I’ll be happy to pass them on to our ML team, feel free to DM me 🥳 (and if you would like more news like this I have a cool newsletter 😎).
*Thread Reply:* Ol Pejeta is great, they're involved with several conservation tech initiatives. I helped out with an aerial census in 2020 for Savetheelephants just before lockdown. We flew thermal + viz cameras in a cessna, part of the novelty was assessing thermal for dawn/dusk. I was stunned to learn that digital imaging is still not the norm for surveying and that manual counts from the back of a plane is more common. I got a ride back over the Rift Valley with the passenger door off (because the cameras were mounted there) which was an interesting experience.
Photo is refuelling at the end of the day. Interestingly Ol Pejeta has an airstrip inside the park where tourists pay lots of money to fly into (there's also a larger airport in Nanyuki nearby). One of the few places where "elephant on the runway" is a valid reason to delay takeoff.
*Thread Reply:* was about to say the same thing: we work with ol pejeta and i have been lucky enough to have been there. very cool to see more tech helping out there -- congrats!
*Thread Reply:* Agreed, great folks there no doubt. ℹ️ We are working on packing the model and shipping it to Mbaza so it can be used. Soon 🙂
*Thread Reply:* Hi Andrzej - This is cool, congratulations! We are working on a similar tech, which is being deployed in two tiger reserves in India. Maybe we can connect and exchange notes.
*Thread Reply:* Cheers @Aakash Gupta, and sure! Let's talk. Sent you a connection request via LinkedIn 🤝
*Thread Reply:* Hey Folks, me again!
Mbaza 2.1.0 with the Ol Pejeta model (described in the blogpost above) added is live! The installer links and instructions can be found in our GitHub.
We'll move on to helping Ol Pejeta team to make the best use of the tool, and later focus on bringing more features (andupdate manuals, training materials and so on, it's about time 😮💨). Anyway, stay tuned, exciting stuff on the horizon, but as usual, lots of work to be done first 🤞.
*Thread Reply:* Hi Jaroslav, maybe contributing to such as challenge would be an interesting option for you?! https://www.fruitpunch.ai/challenges/ai-for-european-wildlife
*Thread Reply:* Hello Magali! 👋 Thank you for the suggestion! I would rather find a project for longer cooperation, but I will definitely check fruitpunch! 🙂
*Thread Reply:* Jaroslav - one of the most exciting companies in the world is hiring in this space... https://basecamp-research.homerun.co/?lang=en
*Thread Reply:* What type of annotation? bounding box?
*Thread Reply:* I used https://github.com/opencv/cvat to annotate turtles. But after annotating 1000 turtles, I saw them whenever I closed my eyes. Good luck with leaves 😄
*Thread Reply:* @Felipe Parodi, yes I was thinking of using bounding box!
*Thread Reply:* i’ve found roboflow to be a great (free) bbox tool, and they have an AI-assisted labeler (once you label a handful of imgs)
*Thread Reply:* If you have an edu account, look at Segments.ai they some neat assisted tools which are free for students/academics.
CVAT is also decent for free software - it's not as polished as the commercial offerings in my experience, but it's fine for bounding boxes. See also label studio, etc.
*Thread Reply:* if you are working collaboratively, there's a number of web based solutions where you can have a lot of annotators operate on the same dataset, such as label-studio (https://labelstud.io/ ) we used that in the past and it was quite useful, although setting up the task has a bit of overhead - definitely worth it for large datasets and many annotators
*Thread Reply:* I also use label studio.
*Thread Reply:* Out of pure curiosity, can we see a sample image with annotations?
*Thread Reply:* what resolution can you see individual leaves? We've been talking about identifying trees from drone imagery to supplement ground data.
*Thread Reply:* I recently needed to do some viewing and editing of bounding boxes in a simple, local app, and landed on this one:
https://github.com/mfl28/BoundingBoxEditor
I've been really happy with it. I think if I was doing a Big Serious Project I would second everyone's vote for Label Studio, but to just quickly edit a small number of boxes, and/or to preview a large number of boxes, BoundingBoxEditor has been quite good.
*Thread Reply:* We used LabelBox in the past (fine for small projects otherwise it gets costly), but during the last two years, we used only CVAT.
*Thread Reply:* CVAT is a great tool for polygon, polylines, bounding boxes and keypoint annotation. If you only need Bounding boxes, then LabelIMG is a great free tool for that. Incase you need someone to do the manual annotation I can help with that as well as that is my area of expertise.
Hello! 👋 St. Patrick's Day greetings to you all☘️☘️
My name is Aakash Gupta, I am a climate entrepreneur and an AI4Good advocate. My team has developed an AI-enabled platform for bio-diversity estimation of the wild population. The platform is being deployed in two wildlife sanctuaries in India; viz. Kawal Tiger Reserve and Amrabad Tiger sanctuary. The system has processed more than 3.85mn camera trap events spread over 4 census years
This project has given me the opportunity to interact with a number of decision-makers and wild-life conservation experts in India and abroad. And their vision and passion for wildlife conservation and adopting new technology has inspired me. Look forward to meeting more like-minded researchers, entrepreneurs, and enthusiasts.
Hi everyone!
My name is Chris Lang, I am a software engineer at the Benioff Ocean Science Laboratory working on a handful of applied marine science projects including some using machine learning (vision).
We're hiring a Software Data Engineer to support the Clean Currents Coalition and other marine technology projects at the Benioff Ocean Science Laboratory at UC Santa Barbara. Learn more about this opportunity or share it with your network here [https://recruit.ap.ucsb.edu/JPF02400]. The estimated salary range for this position is $90,000 up to $120,000. Applications are due April 7th, 2023. UCSB is an AA/EOE, including disability/vets.
*Thread Reply:* The website incorrectly lists the anticipated start date as February. It will now be late May-June but will still be negotiable.
*Thread Reply:* We were just talking about this yesterday to add to a grant proposal, post if you find something. (@Sara Beery). Can you describe what you hope to do with it, just for me to have a sense of workflow.
*Thread Reply:* I know Tom Denton at Google has been doing some flavors of this (with query-based structure) but I don't know that they have anything open source.
*Thread Reply:* @Holger Klinck, do you and the BirdNet team do anything along these lines?
*Thread Reply:* Yeah, a little with Tom. But nothing open source yet - first we need to generate some reasonable results :)
*Thread Reply:* I've been using some distance based queries lately that are working fairly well. Feel free to message me and I will share some of the initial stuff I have running. There is also the ScaNN and Milvus libraries.
*Thread Reply:* @Ben Weinstein Currently, I am dealing with bird species identification in a subset of my recordings for a paper. And it's overwhelming (particularly for a place like Delhi which has so many bird species and lots of them have very large repertoire). I am also planning to do a fine scale annotation in those subset for some ML based work. I know Kaleidoscope (Wildlife Acoustics) has such clustering function. But don't know about any open-source for such utility. So, for my use, it can help with two things; 1) to estimate a direct count of sonotype richness in a sound file and 2) fine-scale manual annotations of those sounds pattern which can be further used for developing a deep learning model for species identification.
*Thread Reply:* Maybe some of the works of Christian Bergler (e.g. Animal-spot) may be worth having a look at: https://github.com/ChristianBergler
*Thread Reply:* @Vincent Christlein Thanks for sharing.
*Thread Reply:* Interested to follow this. In the past for similar problems I've used something like YAMNet or PANNS for feature extraction, UMAP to reduce dimensions, then clustering with affinity propogation where you don't need to designate the number of clusters, in theory the clusters should group similar sounds. Sure there are more up to date/bespoke approaches like those shared above!
*Thread Reply:* What allometric equations are you using? Is the intention to calculate AGB for a forest carbon credit?
*Thread Reply:* We did a carbon study to compare with FRELs. I’m using Chave Model Type II - Tm
ln(AGB) = a + b1ln( D )+ b2(ln( D ))2+ b3(ln( D ))3+ d ln(q)
Diameter (D) in cm and density (q) in g/cm3. The parameters for a and b are also provided in the attached paper.
Happy to review it anytime!
*Thread Reply:* Do you know how the FREL was calculated in Colombia, most likely through interpolations from satellites I almost assume? Did they use Chave et al (2005)? Maybe there are factors they integrated such as additionality etc into their net emissions? We have consistently met discrepancies with reported government data when using our own AI models. Many reasons are political.
*Thread Reply:* The FREL estimates site Alvarez 2012. It appears they are using field sampling rather than satellite data. I’ve run the models for both Chave 2005 and Alvarez 2012 (using the provided best fit parameters) and get vastly different outputs (from each other, and the national average). I’m starting to feel the models don’t hold up…
*Thread Reply:* Regarding AGB you can do following satellite-based sanity check and compare the aggregared AGB from Global Forest Watch: https://data.globalforestwatch.org/maps/e4bdbe8d6d8d4e32ace7d36a4aec7b93 Spawn, 2020: https://www.nature.com/articles/s41597-020-0444-4 Santoro, 2021: https://essd.copernicus.org/articles/13/3927/2021/essd-13-3927-2021-discussion.html
Their data is available publicly and globally and from my experience, for areas > 5ha, Santoro has been surprisingly close to our field-based experiments in Ecuador: https://arxiv.org/abs/2201.11192
This could help you distill a better signal maybe!
*Thread Reply:* I'd start fine-tuning a pre-trained model on your classes. With 50 images, you should augment a lot otherwise, you will overfit quite quickly
*Thread Reply:* Thanks Vincent, do you have a suggestion for a pre-trained model I can use? What do you think about I3S?
*Thread Reply:* what are you referring to? https://github.com/daniel-brenot/I3S-Interactive-Individual-Identification-System-Desktop ? I never worked with that
*Thread Reply:* How distinct are the markings between the individual cats? If you could quickly tell with the human eye, maybe something simple such as Hu Moments or basic PCA classification could be worth looking into.
*Thread Reply:* For now, I would suggest looking at IBEIS https://github.com/Erotemic/ibeis
*Thread Reply:* Also in doing that search I was surprised by how many unannotated data sets are out there, just orthomosaics that someone created in the process of doing a manual count survey, typically from USGS or US/state F&W groups. I did not include those, but those would be fun to index as well.
*Thread Reply:* maybe a bit of a stretch but I would count drone/aerial images for wildlife detection as EO data. So one could add it to the GRSS EOD
*Thread Reply:* I'd say it's helpful to have a separate list for this, since it's a bit specific. Thanks Dan!
*Thread Reply:* This is great-- thanks for putting this together! I may have some more additions after completing the survey of the RS/ML Community of Practice group.
*Thread Reply:* Now I'm getting greedy, and I'd like to not only have a list, I'd like to get a slightly structured record for each dataset, with at least species, resolution, number and type of annotations, etc., plus a couple sample images, and - here's the stretch goal - a couple images where we've rendered an annotation onto an image (to make sure it's actually possible). It's fine if this is literally done manually (e.g. by opening the annotations in a text editor, and drawing a box on the corresponding image in Photoshop), but we can also live without this last step.
This is a bit of a pain, but if we were to divide and conquer, we could do this pretty quickly (like <1 hour), so... reply here if you're willing to sign up to do this for either a specific dataset on the list, or for N datasets that I randomly assign you.
If you're the owner of a dataset and you already have the answer, you still get all the Karma if you volunteer to provide this data for your own dataset!
If I get some takers, I'll send out a link to a spreadsheet.
*Thread Reply:* I'd like to volunteer for this. Let me know what needs to be done.
*Thread Reply:* I would like to volunteer as well for one or two datasets.
*Thread Reply:* Oh this is great, that's 5 volunteers, I'm almost off the hook for doing anything now. 🙂 I'll DM all of you if I don't have your email address, and send "assignments" by email. Thanks!
*Thread Reply:* OK, mission accomplished, we huddled virtually offline and gathered standardized metadata, standardized sample code, and sample annotated images for all the datasets on that list:
https://github.com/agentmorris/agentmorrispublic/blob/main/drone-datasets.md
Thanks to @Zhongqi Miao, @Aakash Gupta, @Edward Bayes, @Kalindi Fonda, and @Josh Veitch-Michaelis for participating in this experiment.
Let us know what the list is missing! And if you're the owner of any of the datasets (I'm looking at you, @Ben Weinstein), let us know if we got anything wrong.
<a href="https://github.com/agentmorris/agentmorrispublic">agentmorris/agentmorrispublic</a>
*Thread Reply:* We are planing to also open source our data from our research project BAMBI in future including drone videos of wildlife such as deer, red deer, boar, chamois, … But this will probably still take some months. I will definetley come back to you regarding that, if you are interested 🙂
*Thread Reply:* Definitely, we will want to integrate that in our airborne work.
I just now got back from a project in Zambia to test a system to mitigate human-elephant conflicts. The rollout was successful and we already saw it in action when we were there. You can get an impression here: https://www.linkedin.com/posts/tsuijtentechforgood-wildlifeconservation-conservation-activity-7047177106161602560-OBhv?utmsource=share&utmmedium=memberdesktop|https://www.linkedin.com/posts/tsuijtentechforgood-wildlifeconservation-conservation[…]106161602560-OBhv?utmsource=share&utmmedium=memberdesktop
*Thread Reply:* I really curious to get feedback from people who also have a lot of experience working with elephants.
*Thread Reply:* Currently we are using our AI-camera to detect the presence of elephants. But I would be really interested in other ways to detect elephants. For instance using audio: detecting their sounds / rumbles. Are there any experts here in that field? Detecting elephants by sound? 🎤 🐘
*Thread Reply:* there is quite a lot of work on different frequencies of calls and vocalizations of elephants. I can reach out and see if they have done detection.
*Thread Reply:* You might want to check this out: https://elephantlisteningproject.org/
*Thread Reply:* Thanks @Patryk Neubauer so you happen to know anyone there?
*Thread Reply:* Know is perhaps a big word 😄 , but Daniela Hedwig from ELP is a mentor on a project I'm contributing to
*Thread Reply:* Joyce Poole with Elephant Voices developed the Elephant Ethogram and has worked extensively on elephant acoustics!
*Thread Reply:* i have only played with this a little bit (personally, not professionally) but it seems to have a decent user base and track record.
*Thread Reply:* Hey that looks promising thanks!
*Thread Reply:* Also iNaturalist's upcoming Seek v2 app that uses React Native could be a candidate to modify
*Thread Reply:* The mobile version of AI2's EarthRanger (https://www.earthranger.com) can do all of this. In offline mode, it will record your location/photos/observations/notes in a local cache and once you have access to an internet connection, the app will transmit your data to a main server. It's easy to retrieve the data afterwards through their Python API, and the app is highly customizable. We're currently using ER Mobile for field census in a 100% offline setting in the Canadian High-Arctic (Nunavut). If you are interested, let me know and I can connect you the right people/share how we've adapt our EarthRanger server to our needs
*Thread Reply:* The CitSci mobile app (https://citsci.org/apps) might be worth having a look at? I can't find "how much" open source is though...
*Thread Reply:* Beat officers in India use NoteCam to capture the lat/lon locations during patrolling activity. As well as to take images of waterholes, pugmarks, PIDs etc. you can attach a note to that image which can then be exported as a KML file. But this is a freemium tool.
Join me tomorrow for an https://www.linkedin.com/feed/hashtag/?keywords=aws&highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7049801509106040833|#AWS Heroes in 15 session on Bear Conservation with ML, Serverless and Citizen Science.: https://www.linkedin.com/video/event/urn:li:ugcPost:7038608935079043073/
*Thread Reply:* There's also an interview by AWS Developer Advocate, Linda Haviv, from AWS re:Invent 2022: https://youtu.be/sKGau7c53go
*Thread Reply:* Maybe it is not exactly what you are looking for but take a look at https://www.kaggle.com/datasets/emmarex/plantdisease
*Thread Reply:* plantnet is a great app/group. maybe this is useful? https://github.com/plantnet/PlantNet-300K
*Thread Reply:* Segmented trees: https://github.com/norlab-ulaval/PercepTreeV1
*Thread Reply:* More generally, most of these don't have segmentation labels, but... a list of image datasets that have something to do with plants:
https://lila.science/otherdatasets#images-plants
*Thread Reply:* Thank you all for the pointers!
*Thread Reply:* Hey Silvia, we have collected quite a huge dataset containing alpine plants from the Nationalpark Hohe Tauern (Austria). It is currently a closed dataset, but our project partner was interested in open source it. I could make contact, if you are interested in that :)
Some side information on the dataset can be found in our publications: https://doi.org/10.5220/0011607100003417 and https://doi.org/10.46354/i3m.2022.sesde.006
*Thread Reply:* Thank you very much! Let me talk with my collaborators, I am not sure these alpine plants are the species we need to consider.
*Thread Reply:* Recently, I came across the BAMBI project. They aim at species monitoring using UAVs. Unfortunately, the website is in German because it is an Austrian project, but maybe you can contact the project leads.
*Thread Reply:* Hi Alayna,
we are using DJI M30T drones for animal detection in our project BAMBI as mentioned by Dimitri Korsch. So we can definitely talk! :)
*Thread Reply:* Hi @James Farrell, nice to see you on here I've also done some work for Klima and I'm familiar with the work you guys have done with Toucan in tokenizing carbon credits!
I was just curious what were the types of AI/Conservation projects you were hoping to undertake that applies to your work in ReFi? Maybe there's an opportunity for collaboration with myself and the other folks in this space 👍
*Thread Reply:* One of my colleagues made an extensive research on XAI libraries. You can contact him. Maybe he could also help you in your task. Here is his blogpost with all his findings: https://www.reasonfieldlab.com/post/a-complete-guide-on-computer-vision-xai-libraries
*Thread Reply:* thanks @Maciej Adamiak, will do! 🙂
*Thread Reply:* I hope the data from the survey (anonymized) will be shared with the participants?
*Thread Reply:* Also you should add "I have not heard of this tool" as an option 🙂
*Thread Reply:* @Aakash Gupta I’m happy to share the anonymised data 🙂
*Thread Reply:* fair point, the idea was that you don’t have to click on each of the rows, so by not selecting an answer for that row it’s implied that the participant hasn’t heard of the tool
📢 Register for the
👉 Learn more and register at bit.ly/ci-2023-rc-eds!
🐦 Like, RT, and tag others on Twitter to help us promote!
https://twitter.com/Climformatics/status/1651007755270905856
*Thread Reply:* maybe - https://zenodo.org/
*Thread Reply:* Depending on the scope of your software maybe the MDPI Software Journal or Elseviers SoftwareX. Both are journals intended for Tool-related publications 🙂 Additionaly, maybe also a Nature Scientific Report. But all three are still scientific, so a pure software documentation won‘t workout 😄
*Thread Reply:* If the software is open source: https://joss.theoj.org
*Thread Reply:* thank you! yes, it is open source
Hi everyone! Just wanted to plug that @Carly Batist and I released another update to the Conservation Tech Directory! We are up to 836 resources in the directory, so if you haven't checked it out yet (or in a while), head on over! As always, if you know of any additional conservation tech resources that aren't in the directory yet, you can fill out our google form and we'll get them added. And if you aren't subscribed to our email alerts, head over to the website so you can be notified any time we release an update! AND we've now reached site visitors from 100 countries!! 🎉
*Thread Reply:* Oh very cool! I work for a professor at UBC on her “Smart Earth Project,” where we are doing a similar compilation and database production using web scrapes; would love to connect with you and chat more about what you’re up to if you ever have some spare time.
*Thread Reply:* Thanks for sharing, this would be an awesome resource!
We're looking for a seasoned Research Director to join our growing Earth Species Project team and lead our long-term ML research agenda into the communication systems of other species. Please share with your networks! https://lnkd.in/gQpHizXJ
.jpg)
*Thread Reply:* My experience with one recently (MetaFormer) is that they work. I don't know about "big margin" because frankly we didn't test on other architectures and we wanted something quick and dirty. In this case we picked the architecture that claimed SOTA on some tasks and tried it (and it certainly works well, even with 5k classes). To a great extent it depends on dataset quality and the usual how unbiased your evaluation is (e.g. checking that your test data are balanced and one easy class isn't skewing your results)
Did it work? Yes. Was it a pain because of hardcoded choices the researchers made in their repo? Yes 😅
A strong recommendation is to use HuggingFace's model zoo, they have "reference" implementations of a lot of this stuff and I think it's probably easier to use that than hack somebody's conference submission that only works on benchmark data. Inputs and outputs are well defined and the code is mostly well-tested. They also provide training examples for most of their models here https://github.com/NielsRogge/Transformers-Tutorials/tree/master
I think another interesting avenue for ecology that is under-explored is better pre-trained models from e.g. drone observations, camera traps, etc. DINO2 had some pretty good boosts for satellite analysis that way.
*Thread Reply:* I can share my specific experience with agent base model I developed a while ago. there is the ODD protocol developed a specially for such model because of the complexity to explain the agent environment interaction. I also worked on a pipeline for protein comparison, which i didn't find any protocol for (back at that days), but for sure - there where no similarities between the processes, well aside the IDE and the Python environment 🙂.
for further details about ODD: https://www.jasss.org/23/2/7.html
*Thread Reply:* Hey Ben, I found this post by Karpathy extremely helpful -- http://karpathy.github.io/2019/04/25/recipe/
*Thread Reply:* The paper "Perspectives in machine learning for wildlife conservation" by Tuia et al., 2022, may be helpful: https://www.nature.com/articles/s41467-022-27980-y.
*Thread Reply:* Google has a fairly technical doc about this (on top of Karpathy's essays which are usually great - as well as his Zero to Hero series)
https://www.nature.com/articles/s41467-023-38901-y
*Thread Reply:* Reminder: Today is the last day to apply for Image Datapalooza!
*Thread Reply:* Have you checked the LILA dataset.
Hi all, I'm guest editing a special issue of Remote Sensing on remote sensing applications in biodiversity conservation and ecological modelling, which is accepting submissions until October 9th 2023. If you are interesting in contributing please feel free to message me directly or check out the special issue link (https://www.mdpi.com/journal/remotesensing/special_issues/8OQ6QPGDQ2). Thanks! 📡🌏
*Thread Reply:* Also, if you aren't attending CVPR but you are in the Vancouver, BC area, feel free to join in and meet up as well!
*Thread Reply:* Also crowdsourcing locations for an informal happy hour that is good for groups!
*Thread Reply:* Visited these places with large groups last time I was in Vancouver a few years ago
- https://goo.gl/maps/upccLW9RKWe1jVQC6
- https://goo.gl/maps/nN9UZbU7TJvngNmk9
- https://goo.gl/maps/Hv8wuJcTQ8ut2c3GA Earls was great. That would be my top recommendation. But things might have changed quite a bit over the years.
Savimbo just released an indicator-species biodiversity crediting methodology for review that allows indigenous groups to sell direct to climate markets, by calculating their environmental credits on Google Earth Engine. We are looking for developers to help us review and open-source this code. Please dm me if you want to be a technical reviewer. Its gone to Cercarbono, Verra, and Plan Vivo for certifying body adoption.
Have been thinking/doing/investing at the intersection of biology and AI - here's a synthesis of the story so far: https://www.linkedin.com/posts/georgedarrahthe-key-to-natures-intelligence-is-artificial-activity-7075478440811884544-IFDm?utmsource=share&utmmedium=memberdesktop|https://www.linkedin.com/posts/georgedarrahthe-key-to-natures-intelligence-is-artifi[…]440811884544-IFDm?utmsource=share&utmmedium=memberdesktop
@Aamir Ahmad thanks again for inspiration re the zebras!
Who wants to try getting a 3D dog model from an RGB image? Try out our demo here: https://huggingface.co/spaces/runa91/bite_gradio
Hi Climate enthusiasts,
There is a VC that I know of that is rolling out a grant funding initiative, specifically focused on synthetic biology and climate.
The grants are fast grants (30 min application, 21 day turn around and up to 100k per project) with no IP taken and no reporting.
They are looking for BOTH applicants (project proposals) AND PIs to join their expert review panel.
Would anyone in this extended network be interested in being an expert reviewer on this climate initiative? Please forward widely. You can participate even if your general focus is simply climate!
Professors can both be reviewers and apply for the grants...AND many students per lab can apply.
Happy to directly connect any interested PI's with the program leads.
Warmly, Giana Cirolia
See below for full details:
As mentioned, Manifest Grants is a 'fast grants' program. We award $25K - $100K to scientists to prototype their most ambitious synbio ideas for solving the climate crisis. We're looking for senior reviewers for the June-July period.
Expectations: • Review ~10 applications. • Review time ~ 40mins per application. Fully numeric scoring system, no write-up required! • Get paid $30 per application! • Submit the review within 8 days. • Sign an NDA to protect the confidentiality of the proposed ideas. • You can apply even if you participate in the review! Interested? Here's my calendar if helpful.
P.S Our first iteration was with Repro Grants. We received 465 applications from all the top universities and funded 12 projects in female reproductive science. Read more (Forbes article). Sara
— Sara Kemppainen Fifty Years Linkedin / Twitter
*Thread Reply:* Thank you all to the initial interest! Please keep Direct messaging me :)
*Thread Reply:* Hi @Giana Cirolia, can you provide a link for the funding applications/submissions? Are these grants only available to academics / in the US, or can NGOs apply for some projects? Thanks!
Hi all! I'm new to the group and just wanted to throw a quick intro on the channel.
I'm a current postdoctoral fellow at the University of Southern California looking at how we can use CV to rapidly mobilize historical biodiversity data in order to reconstruct past distributions, phenologies, and morphologies. The end goal is understanding how global change processes are impacting biodiversity on the planet and to translate these data and reconstructions to on-the-ground conservation organizations looking to conduct species' assessments and influence policy. I largely do this work with butterflies but am interested in all other groups as well (especially other invertebrates!).
Looking forward to being here and learning more! BTW, you can find me on Twitter, LinkedIn, or ResearchGate if you'd like to chat or feel free to shoot me a DM. :)
*Thread Reply:* Not sure if I fully understand your ask, but this paper is a great overview of the ML space within biodiversity! https://www.nature.com/articles/s41467-022-27980-y
*Thread Reply:* Likewise not sure if totally got the ask but this paper seems relevant too https://gpai.ai/projects/responsible-ai/environment/biodiversity-and-AI-opportunities-recommendations-for-action.pdf
*Thread Reply:* Thanks @Carly Batist @Rajiv Pattni recommendations are useful.
Hi Everyone, Just thought I’d briefly introduce myself. I work as a software engineer at NatureMetrics primarily working on backend systems and data pipelines. I have a keen interest in ML engineering and AI in general. I hope to learn more from everyone in this community about how this can be applied to Biodiversity. If you have any interesting opportunities or would like to collaborate please reach out! (Particularly interested in any research opportunities available) Thanks! Sonny 🙂
Hi everyone!
My name is Jacob and I'm an ML engineer and developer evangelist at an open source cv/ml company Voxel51. Before this, I completed my PhD in physics, and I'm really excited about applications of AI for climate and conservation. Thanks @Sara Beery for letting me join this awesome group!
Three Opportunities:
- I help run a virtual computer vision meetup with 4,500+ members. If anyone here would be interested in speaking at one of our events, please let me know!
- I write on Medium, and I'd love to highlight some of the incredible work this community is doing. If you are using CV in climate/conservation and want me to write about it, feel free to reach out 🙂
- The company I work for helps make datasets more accessible - we worked with authors of a bunch of CVPR datasets to make their datasets publicly browsable at cvpr.fiftyone.ai. If you want to make it easier for others to view and use your datasets, please reach out! Thanks so much, and so excited to connect with the amazing people here
📢 Seeking your support to attend Deep Learning Indaba in Accra, Ghana! 🌟
Hi everyone! 👋
I wanted to reach out and ask for your support in attending the Deep Learning Indaba in Accra, Ghana. As an Omdena chapter lead in Botswana and a member of Sisonke Biotik, an AI and healthcare community in Africa, this conference presents a fantastic opportunity for me to enhance my AI skills and contribute to the development of AI in Africa.
The Deep Learning Indaba brings together some of the brightest minds in AI and offers a platform for learning and networking. By participating in this event, I'll have the chance to connect with experts and passionate individuals, accelerating our efforts to leverage AI advancements in Africa and improve access, quality, and equity in AI development.
I kindly ask for your support by considering a donation towards my journey. Every contribution, no matter the size, will bring us closer to unlocking the potential of AI for the betterment of Africa.
Please donate here: https://gogetfunding.com/help-ditiro-attend-deep-learning-indaba-in-accra/
Additionally, I would greatly appreciate it if you could share this message in your networks to help spread the word. Together, we can make a lasting impact and create a brighter future through the power of AI.
Thank you for your support and belief in the transformative power of AI. Let's seize this opportunity and drive positive change in Africa!
Best regards,
Ditiro Rampate
*Thread Reply:* Hello guys👋:skintone5:, I am still raising funds to attend the Deep Learning Indaba this September in Accra. I also have fundraising posts in twitter and LinkedIn and would appreciate if you could not only but share among your networks for better reach.
*Thread Reply:* Hi Dan, welcome! You can join the #marine channel for discussions regarding underwater computer vision 🐟 📷
*Thread Reply:* See you soon at the drone workshop, actually!
*Thread Reply:* Just detection or re-ID too? We're working on multi-species big cat re-ID models and were going to include the lila.science dataset as well. Happy to collaborate. Feel free to DM directly.
*Thread Reply:* Hi Jason, thanks so much for coming back to me. Just detection initially but we'd like to explore re-id in the future! I'll DM you. Thanks so much! 😊
*Thread Reply:* I can't help you with "tiger-specific datasets", but if I grab the giant .csv file that contains taxonomic information for all the camera trap data on LILA:
https://lila.science/taxonomy-mapping-for-camera-trap-data-sets/
...and I grep for "panthera tigris", I get 321 hits. Not a lot, but that's something. If you download that big .csv file and run:
cat lilaimageurlsandlabels.csv | grep -i "panthera tigris" | cut -d "," -f 2
...you should get a flat list of URLs that's easy to download, e.g.:
https://lilablobssc.blob.core.windows.net/wcs-unzipped/animals/0462/1185.jpg
Tigers are an interesting case where basically all of the candidate data comes from areas that (for good reason) have historically been hesitant to allow data publication, even images, so data sparsity is a bit fundamental to your problem. Maybe 321 wild camera trap images plus the handful of zoo images is good enough to get you started... does the organization you're working with have any labeled images, and/or do they think they have tigers amongst a pile of unlabeled images?
Also when you say "fine-tune MD", confirm that you mean "add species (or tiger/non-tiger) classes to MD", as opposed to "make MD more accurate"? There are lots of situations where fine-tuning to make MD more accurate would be worth it, but my guess is that MD is probably doing asymptotically well on tigers: they're big, they're patterned, they look like lots of other species MD saw in training, and we did have a bunch of tiger data from the Wildlife Institute of India during training.
*Thread Reply:* And of course, there's always the time-tested strategy of begging Ecology Twitter for data.
*Thread Reply:* Hi Edward, Here is a fine scale model for MegaDetector (https://github.com/bhlab/SpSeg) that works really well in segregating tigers (>98% accuracy). This model was developed using datasets from Central Indian landscape, so it would work fine for Bhutan as well.
Unfortunately, training dataset isn't available for public use.
*Thread Reply:* Thanks so much Dan and Shivam!
@Dan Morris - really useful feedback, thanks so much! I’ll find those 321 images in the LILA dataset! And totally hear your point re: data scarcity. They do have labeled data, yes, but to exactly your point there are sensitivities in sharing such data, particularly outside the country, so before navigating that thorny issue, we wanted to see how performant we could get a model on publicly available data. Re: fine-tune, I mean classes yes, not improving accuracy (using your super help tutorial as a start https://www.kaggle.com/code/agentmorris/fine-tuning-megadetector)
@Shivam Shrotriya - thank you so much! It sounds like this might be exactly what I was looking for. Thanks again!
*Thread Reply:* Definitely let us know what you learn re: SpSeg, fine-tuning, etc.
The only minor thing I'll correct here - in the interest of giving credit where credit is due - is that the fine-tuning notebook you linked to isn't really mine, I just forked it temporarily to make minor copy edits that were pulled into the upstream version. The original version is here:
https://www.kaggle.com/code/evmans/train-megadetector-tutorial
It is a really nice tutorial, but I can't take credit for how nice it is. :)
*Thread Reply:* So a suggestion for such long tailed classes. Is to run an image search on google/Bing or other search engines. It will give you a large set of images, which you can simply download and use for training.
*Thread Reply:* I workwith the Indian government for deploying an AI platform for processing and storage of CT images. A lot of of forest officers in India are on twitter. And they keep posting wildlife images. A search on twitter should give you some datapoints. DM me, if you would like to talk.
We (Skylight, AI2) are hiring an MLE to help build maritime intelligence applications for ocean conservation: https://boards.greenhouse.io/thealleninstitute/jobs/5171288 #AI4Good
*Thread Reply:* Hi @Patrick Beukema, each S-2 scene has metadata in the GeoTIFF file which contains the coordinates of the upper left pixel of the image, and the resolution of each pixel. These two are enough to generate the coordinates of each pixel in the image. If you load the file using rioxarray in Python you already get the coordinates in the dataset array.
*Thread Reply:* hey thanks yeah I am familiar with that method, my goal is to build the most precise coordinates possible. Do you know what the precision of that method is and if its optimal?
*Thread Reply:* Reading the scene metadata, as I suggested, you are basically assigning the coordinates to each pixel based on the orthorectification that the data provider did. rioxarray or any other package doesn't change the precision of the coordinates and they are as accurate as provided in the metadata.
If you are interested in correcting the alignment of scenes through time, aka mis-registration error, then the problem is something else. There are methods in the literature to mitigate this but I don't know the best ones.
*Thread Reply:* Very cool indeed! Is there a two-sentence summary of what the automated decisions would be? E.g. are you capturing/tagging automatically for survey purposes?
*Thread Reply:* @Dan Morris Right now the system uses the real-time output of MegaDetector to decide whether or not there is an animal in the detection chamber, and if so it continues data collection until no identifiable animal is present. We are extending that to an automated mark/recapture mechanism with the eventual goal of precision-targeted vaccine delivery in the field, down to the individual level.
*Thread Reply:* Let me know if Wild Me (wildme.org) can be a future part of individual ID part.
*Thread Reply:* @Jason Holmberg (Wild Me) Definitely! Looking now at your website. Individual CV ID for small mammals like deer mice and ground squirrels is challenging. I'll be in touch.
Hello Everyone, I wanted to introduce myself to the channel I am Tim Mayer, Research Scientist at University of Alabama in Huntsville and the Regional Science Coordination Lead for the SERVIR Hindu Kush Himalaya region as part of SERVIR global's NASA Science Coordination Office (SCO). SERVIR is a joint initiative of NASA and USAID and we are working around the globe with local partners to develop geospatial solutions and applications to address environmental challenges. I want to thank @Ben Weinstein for connecting me to this group!
Within NASA SCO our team has a dedicated TensorFlow Working Group (TFWG) which focuses on capacity building, application development, and knowledge sharing around Deep Learning approaches. I will be sure to share future talks and updates coming from our group here.
Again really excited to join this slack channel! Tim
*Thread Reply:* Perhaps Smarter-labelme https://github.com/robot-perception-group/smarter-labelme can help? It is very useful for tracking objects over time for quick annotation, but you can also use it for an arbitrary set of images. Let me know if you find it useful and we (@Eric Price or I) can help you with any questions about that. The only issue could be multiple channels (other than RGB).
*Thread Reply:* I recently posted about it on Linkedin -- https://www.linkedin.com/posts/ahmadaamir_robotics-labeling-annotate-activity-7086729440344322048-qAud?utm_source=share&utm_medium=member_desktop
*Thread Reply:* Here's a comprehensive list of annotation tools that you can use with their github links: https://www.thinkevolveconsulting.com/list-of-open-source-annotation-tools-for-machine-learning-research/
*Thread Reply:* Have you tried using the segment-geospatial or segment-Anything-eo packages. I have tried it with drone imagery and it seems to give a good output. Maybe it could be trained for Sentinel images.
*Thread Reply:* Thanks so much! I don’t mind if it’s commercial, paid for product. But these are a great reference, thanks!
*Thread Reply:* i work for kitware inc. and am a technical lead for DIVE, an open source video/image annotation platform: https://kitware.github.io/dive/
*Thread Reply:* others in my team have expertise in geospatial data as well
*Thread Reply:* our software is generally open source, and we partner with people to develop customizations needed for your specific workflow
*Thread Reply:* if you're interested in exploring whether we can help you, @Patrick Beukema, please do reach out via DM
*Thread Reply:* @Aakash Gupta Great list! One to add: https://github.com/mfl28/BoundingBoxEditor. It's fast and simple, and the developer has been very responsive. I'm trying to work out a workflow for doing a small number of offline annotations, and FWIW my "top two" right now are LabelMe (the offline one) and BoundingBoxEditor. But I wasn't aware of DIVE, definitely going to take a look at that too!
*Thread Reply:* I use CVAT, which has a free online interface. I have not checked out these other tools, but I will be looking at them next time I need to annotate.
*Thread Reply:* @Peter Kulits @Chuck Stewart @Jason Holmberg (Wild Me)
*Thread Reply:* Sure, people have been using video for animal re-id for a while, though unfortunately most of it isn't public. Here's a handful of papers that release video re-id datasets that I think have the information you're looking for: • Towards Self-Supervision for Video Identification of Individual Holstein-Friesian Cattle: The Cows2021 Dataset • Visual Localisation and Individual Identification of Holstein Friesian Cattle via Deep Learning • Classification and Re-Identification of Fruit Fly Individuals Across Days With Convolutional Neural Networks • Re-Identification of Zebrafish using Metric Learning • One public, one by request: Deep-learning based identification, tracking, pose estimation, and behavior classification of interacting primates and mice in complex environments
*Thread Reply:* Tracking is basically frame-by-frame re-identification and you have the ID's in the ground truth files. So if you find a lack of variation in re-id video datasets of animals for benchmarking purposes maybe an option could be to use tracking sequences as a supplement? Just a quick idea. • 3D-ZeF - stereo tracking dataset with zebrafish • BrackishMOT - tracking dataset from brackish water • Tracking of ants
*Thread Reply:* yes, lots of conservation-related data is still distributed in tabular formats! e.g. the largest occurrence records, GBIF is distributed as parquet, & eBird is distributed as compressed csv. What kind of conservation data did you have in mind?
Hello everyone,
With my original background in ecology/conservation, I'm working on an open science project to give motion-sensing wildlife camera traps binocular vision. The goal is for camera traps to capture spatial information along with wildlife photos. This data has the potential to greatly ease estimating wildlife populations.
I've been working with some engineers from the Gathering for Open Science Hardware, and one of them has devised a 3D-printed stereo lens attachment for camera traps, while not adding to the power envelope. We hope this will be easier than modifying camera traps or building them from scratch to provide 3D vision. I know some of those here are working on stereovision algorithms for wildlife observations, and being able to directly and efficiently obtain stereo images could be helpful.
To that end, we've started a crowdfunding campaign on Experiment.com, and thanks to generous contributors we're already at 92% of our funding goal but with only 9 days left!
I'm posting this here to request your support and spread the word. Any contribution, even just $1 will help! This project may also be of interest as demonstrating a different way of funding open science. If successful, our design of the stereo lens for camera traps and any data obtained will be published as open data and open source hardware.
Let me know if you have any questions/comments, and thank you in advance for you help!
*Thread Reply:* Oh I know that first-author! Nice example
*Thread Reply:* I'd be curious to see if other examples pop up. I would say machine learning is not popular yet with movement ecologists. Here is another example though: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0235750
Also, if you consider MAXENT to be ML, it is frequently used with telemetry data to estimate resource selection.
*Thread Reply:* see also #esa2023happy_hour which is on tuesday, so hope you can make it.
Very happy to announce our latest work in the direction of inferring animal behavior automatically from drone videos. Not only this, we also release a never-seen-before kind of dataset of zebras (both plains and Grévy's) recorded in their natural environment in Kenya using two synchronized drones.
Code: https://github.com/robot-perception-group/animal-behaviour-inference Paper: https://www.biorxiv.org/content/10.1101/2023.07.31.551177v1 Dataset: https://keeper.mpdl.mpg.de/d/a9822e000aff4b5391e1/ Video: https://youtu.be/Zu-t0JJsz5o
Feel free to reach out to me about it.
*Thread Reply:* Awesome! Dan Rubenstein was telling me about this in Anchorage a couple of weeks ago. Do you think it can work in nadir footage of zebras, or is the oblique viewpoint necessary?
*Thread Reply:* Thanks. The network as is will most likely need a small bit of fine tuning, but that is what this paper is all about.. with a small number of manual annotations you can kick start the whole process and have the network tuned for your dataset. If you want, you could send us a few pictures to try out quickly.
*Thread Reply:* If it's not too much trouble it would be really interesting to know how it works OOTB with my data. I'll send a few example frames to your email. Thanks!
*Thread Reply:* No prob! Looking forward to it.
*Thread Reply:* Really neat dataset! I'm able to click through to some of the individual images, but if I try to download the dataset (or one of the top-level folders), I get "size too large". I don't know how Keeper works... do you have a recommendation for downloading the dataset?
*Thread Reply:* Thanks! There is a server configuration limitation apparently (as the folders are about ~100GB). I am trying to get it resolved through the MPI admins. In the meantime, I will figure out another way to share it with you directly.
*Thread Reply:* Thanks! No rush to share with me directly... I'm just wearing my dataset-list-making hat, and part of that is making sure datasets I add to various lists are downloadable. I'll be your first download beta-tester when you get it all worked out.
*Thread Reply:* I would love to get a copy too. Thank you!
*Thread Reply:* Hi @Dan Morris and @Jason Holmberg (Wild Me).. we solved this download issue with a workaround. There is now a zipped version of each folder in the root folder. You can download the zips individually. Downloading the folder was not working apparently because of some buffer size limit issue, but now you do not need to download the folder. However, I leave the folders also there so people can browse through individual files if they want. Let me know if there is still some issue.
*Thread Reply:* Thanks! This is straightforward to work with; I confirmed I could read the annotations and render boxes on a sample image, then added to the running list of aerial/drone wildlife datasets:
While I was there, I also added another one from the backlog, from Koger et al, ~40k boxes on ungulates and geladas in drone images:
*Thread Reply:* Hi Carl! Just pinging @Nathaniel Rindlaub for any insights?
*Thread Reply:* I'm aware of two standards... the Camera Trap Metadata Standard (CTMS):
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5267527/
...was used by eMammal; I'm not aware of current systems that use this as a native format.
If I were putting my eggs in one basket, it would be the newer Camtrap DP standard:
https://tdwg.github.io/camtrap-dp/
...which (according to the page, I've not personally verified this) is exported by Agouti and Trapper, and supported as an input format for GBIF.
Approximately 7000 people, approximately 3000 of whom are on this Slack, wrote a paper about it:
https://ecoevorxiv.org/repository/view/5593/
Given that many of those authors were also authors on the original CTMS standard, I think it's safe to declare the CTMS deprecated.
*Thread Reply:* Welcome to the community, what would you say the largest challenge your team faces? Model development? annotations? We’ve done some wind farm bird monitoring work, just curious.
*Thread Reply:* Probably sufficient annotations and distinguishing near misses from collisions. What kind of challenges has your team faced?
*Thread Reply:* We are on surveys before active energy development, so we have to cover large areas to look for birds in proposed wind areas. Large data sizes and very long species list.
*Thread Reply:* That's really interesting, what are your survey methods? Are you using cameratraps looking up at the sky or drones or something else?
*Thread Reply:* I don't know of any tools that do this, though you could certainly use existing high-resolution (let's say 1m) tree cover maps as training data for a system you'll eventually run with coarser (10m?) imagery. Consider the Chesapeake Land Cover dataset:
https://lila.science/datasets/chesapeakelandcover
...which is really an older version of:
...or the NOAA Coastal Change Analysis Program's high-resolution map:
That's a lot of geographic bias, but it's a place to start. I'm not aware of other large (i.e., larger than one city) high-resolution land cover or tree cover datasets, though I don't claim that I would know of others.
*Thread Reply:* check out the Segment-Anything-model. you can model it to annotate the tree-area or building foot-print and then pipe it to your model with lower resolution imagery
*Thread Reply:* https://www.nature.com/articles/s41586-018-0411-9
A worthwhile read on this topic. You can (and should) replace modelling components, but the workflow is still valid.
*Thread Reply:* There is a good performing model training on the iNat2021 competition dataset now available on HuggingFace. Worth checking out, there is also an online demo on the right of this page: https://huggingface.co/timm/eva02_large_patch14_clip_336.merged2b_ft_inat21
Hi all, we are now back in the Hungarian Steppe to deploy and test our autonomous blimps for biodiversity monitoring (here Przewalski's horses in particular). Feel free to follow updates regarding this on my LinkedIn page (https://www.linkedin.com/posts/ahmadaamir_autonoumous-blimp-day1-activity-7095860403892559872-AZJj?utm_source=share&utm_medium=member_desktop)
https://arxiv.org/abs/2201.11192
*Thread Reply:* PyTorch Dataset available in TorchGeo -- https://torchgeo.readthedocs.io/en/latest/api/datasets.html#reforestree
*Thread Reply:* Hmmm... this is a nice idea but given the known high levels of uncertainty of applying general allometric equations (particularly in the tropics, and particularly for equations without height as an input), I hope others would also search out e.g. TLS or destructively sampled ground truth datasets, which are often available. It would also be great to see these datasets attempt to compute the uncertainties in their biomass measurements.
I do find it disappointing that the title discusses 'tropical forest carbon stock' when this is very specifically a small scale dataset of regional tropical agro-forestry. Monitoring agro-forestry for NBS payments is a very worthy aim in itself, and I think it would help the community progress if manuscripts/datasets were more specific about their scope...
*Thread Reply:* Hi emily, I was part of the paper and unfortunately it's too late to rename but I agree that agroforestry would have been a more appropriate title! good point thank you
*Thread Reply:* Following up here, were all trees intended to be annotated? Or just ground truth trees with biomass? https://github.com/gyrrei/ReforesTree/issues/5
<a href="https://github.com/gyrrei/ReforesTree">gyrrei/ReforesTree</a>
Hi all, we hosted David Rolnick a few weeks ago in our Environmental AI series. He delivered an inspired talk on applying ML for climate action. Among his many incisive remarks: “AI for Good” doesn’t mean just adding new “good” applications of AI. It means shaping all applications of AI to be better for society. Check it out! https://www.youtube.com/watch?v=rs4MpjNxOLQ
*Thread Reply:* Hi Charlotte! I don't do any NLP work, but I went to Harvey Mudd and work in bioacoustics 🐋
*Thread Reply:* Hi Charlotte! My background is primarily in NLP and I've done some work in the AI x Conservation space (cv). Happy to chat more if I you think i'd be a useful resource! :)
Hi Guys, just wanted to update on our fantastic field trip!
Our airship was able to visually track and autonomously follow Przewalski's horses roaming free in the Steppe. In the video below, we show how the blimp 'fights' the wind to keep the horses centered in its field of view. We are a step closer to do long term monitoring with autonomous robots. Some more info in the post here: https://www.linkedin.com/posts/ahmadaamir_hungarian-steppe-airship-activity-7099298534696312832-EC-k?utm_source=share&utm_medium=member_desktop . Congrats @Eric Price and @Pranav Khandelwal for an awesome teamwork this summer!
*Thread Reply:* From what I understand, the relation between sat image and yield is highly crop specific, so some reference ground data is necessary to calibrate the estimation for a specific type of crop, but there's quite a lot of work on that topic: https://scholar.google.com/scholar?q=related:2qpIflWO5X8J:scholar.google.com/&scioq=estimating+crop+yields+from+satellite+images&hl=en&as_sdt=0,5
*Thread Reply:* Hi is it okay if I share this in the Explainable AI slack channel that I am a part of??
*Thread Reply:* I'm currently a PhD student but super interested in the project topic. Would love to chat sometime about what you are working on 🙂
*Thread Reply:* There is @Devis Tuia https://join.slack.com/t/explainableaiworld/shared_invite/zt-21xy5r0wm-N4XjItBxoCsExFDsWFdvVw
Hi everyone! Two upcoming postdoctoral fellowship application deadlines:
- The first is the Christopher W. Clark Postdoctoral Fellowship in Conservation Bioacoustics at the K. Lisa Yang Center for Conservation Bioacoustics at the Cornell Lab (revised deadline October 15). The selected candidate will be the inaugural recipient of this three-year fellowship. Candidates will propose innovative and independently developed bioacoustics applications in service of conservation, honoring the legacy of the fellowship’s namesake and program’s founding director, Christopher Clark. While Chris Clark primarily focused on baleen whales, scientists at the Yang Center apply bioacoustic methods for studying marine mammals, fishes, birds, primates, frogs, bats, elephants, insects, anthropogenic sounds, and more in ecosystems around the world! Yang Center staff collect and synthesize data, develop hardware and software tools, and work with partners and students around the world to facilitate the use of bioacoustics in conservation. We encourage proposals with bold, potentially transformative research potential, and especially welcome projects that leverage the tools or research developed at the Center to have an impact on biodiversity conservation (e.g., tool development, knowledge generation, implementation etc.).
In addition to salary support and annual raises, applicants can anticipate approximately $10K/year in research support, with the possibility of additional funds through collaboration with ongoing projects at the Center. For the full posting and to apply, please visit: https://academicjobsonline.org/ajo/jobs/24693.
In the cover letter, applicants should suggest one or more members of the Center as a potential postdoctoral advisor. This person is not required to be a direct research collaborator, but will provide administrative support, professional guidance, and a connection into the Yang Center and Lab of Ornithology. Any member of the Center can serve as a secondary or co-mentor. Primary advisors can include: Dena Clink, Ben Gottesman, Daniela Hedwig, Holger Klinck, Aaron Rice, Larissa Sugai, Laurel Symes, and Connor Wood. Please reach out to us with questions!
- The second upcoming fellowship is the Edward W. Rose postdoctoral fellowship (deadline September 9). This fellowship supports postdoctoral research at the Cornell Lab of Ornithology. We encourage you to learn more about this fellowship and to share the information with your colleagues and research networks. As part of the Cornell Lab of Ornithology, the K. Lisa Yang Center for Conservation Bioacoustics can host Rose postdoctoral fellows who are conducting research that is synergistic with the research conducted by the Yang Center, even if fellows are working on taxa other than birds.
Candidates are welcome to apply to both positions, while noting that the Yang Center Clark Fellowship is particularly focused on bioacoustics and conservation, and that the Rose Fellowship supports positions across the Lab of Ornithology, is open to topics beyond bioacoustics, and can have a focus on basic or applied research.
If your research advances bioacoustics research and conservation, we encourage you to contact us and apply to either or both programs!
*Thread Reply:* Hey @Arvin Sun this is a bit spammy and not relevant to this community. Can you please take the post down?
*Thread Reply:* Thanks @Justin Kay for the heads up
*Thread Reply:* Hi Casey, generally speaking, there is "a lot" more going on in the terrestrial domain when it comes to machine learning and computer vision. A simple reason for that is the hostile environment; it is just much harder to obtain data under water. However, we have a channel called #marine where we post things related to, you guessed it, marine stuff and ocean species! Please, come join us if you are interested in the wet domain 🌊🐟
*Thread Reply:* Unsure I'd be able to give a ball park figure, but marine-focused research makes up a relatively small proportion of work in this domain.
Anecdotally, I am in the process of writing up a literature review of CV applications to benthic environments, and when I discuss future direction, a lot of the papers I talk about focus on terrestrial data rather than from other marine environments. This is mostly due to the higher abundance of terrestrial data allowing for greater research scope.
It's relatively cheap to set up some land-based camera traps and leave them in an area for a few months, but marine data can be expensive to obtain, requires a large human effort, and often needs specialist equipment, especially at depth.
*Thread Reply:* Agreed with above. In terms of number of labelled images that can be used to train detectors for instance, terrestial has several ordered of magnitude more publicly available labelled data.
*Thread Reply:* And even then, quite a bit of the available ocean data is surface-based imagery of animals like whales, turtles, and sharks/rays.
*Thread Reply:* Fascinating discussion for those of us working in the marine space. We get asked all the time “how many fisheries are unsustainably fished” even just a simple count of “how many fish are there” is incredibly challenging. There is an older article from the Atlantic on the topic: http://webcache.googleusercontent.com/search?q=cache:https://www.theatlantic.com/science/archive/2016/10/how-many-fish-are-in-the-sea/502937/ There is a quote in there which stuck with me:
“1970s with John Shepherd, a fisheries management specialist at England’s University of Southampton: Counting fish is like counting trees, but the trees are invisible and constantly on the move.”
This topic is really critical to us, and we would like to do a better job at answering these types of questions, and more broadly linking our work (Skylight) to these metrics where possible. I spoke to
@Cameron Trotter I am very interested in that lit review, if I can help you in any way, let me know.
*Thread Reply:* @Patrick Beukema more than happy to send it your way when I'm able. Would be good to keep in touch if you're working in the space 🙂
*Thread Reply:* yeah that sounds great. maybe we could even set up a talk or something? A lot of people on our team would be interested. Its quite a complex space to model/get good data on with all the different dynamics, human behavior, stats, a lot of noise.
*Thread Reply:* I would be interested in this as well. I am currently working on measuring whether/to what degree trawlers affect the seafloor and marine habitats in the North Sea west of Denmark. Also, for those of you who are interested, I will defend my PhD on September 8. The topic is detection and tracking of fish using computer vision from controlled lab environments to the wild. I will give a 45 minute presentation about my work aimed at a broad audience who doesn't necessarily know anything about computer vision and machine learning (so a lot of images and very few equations). I will put up a zoom link in #marine next week.
*Thread Reply:* Have you checked out fathomnet for undersea images?
*Thread Reply:* @Elizabeth Campolongo I have yet to make use of it, though it is on my todo list. I work with Antarctic benthic imagery however, which is an area that even by marine data standards can be somewhat sparse, so I am unsure if it would help in my specific situation, though it may be a good resource for those who work in more explored waters. I'd be interested to hear from those who have used it in the past for their work, it could be an extremely valuable resource!
https://twitter.com/CstGhosh/status/1695350270958490069
*Thread Reply:* Hey Paul,
Given that the camera is stationary: the keyword here would be background subtraction. We had some experiments with videos, where it works quite well. It depends on the scenario and your requirements if it works good enough with two images.
*Thread Reply:* hey Paul, it seems like a reasonable assumption at first, but there are some caveats.. you need to be able to fly twice (seems that you do) and you need to aqcuire the images athte same time of the day (to avoid shadows or illumination strong changes). You also want to avoid fyling on a sunny day vs a cloudy one (you are in South Africa right?). Finally, you also will need very precise co-registration of every pixel, otherwise you will detect a lot of borders as objects.
*Thread Reply:* on a stationary camera this is ok-ish (as Maximillian says), but you seem to eb working with drones, right?
*Thread Reply:* To add on the previous suggestions: if you have small uncontrolled camera motion, you may need a preprocessing step where you perform 'image alignment' before running the background subtraction process.
*Thread Reply:* Thanks for the swift responses, yes this would be from a drone and I was thinking that with real-time white balance I could correct the lighting for each image. But I do understand that slight variations in height will result in different pixel sizes but currently they are around 5cm so slight difference in height won’t have a big impact on resolution
*Thread Reply:* I'm interested whether drone folks know of anything off the shelf for this... though Devis is definitely "drone folks". 🙂 So the fact that he didn't say "use package X that everyone knows about" suggests it's not an off-the-shelf thing. And it seems like even if you have GPS on the drone, to even try background subtraction, you need registration that's tighter than what you'll get from an off-the-shelf georeferencing tool. Although I say that with no basis in reality, I've never tried.
But assuming you will have to be in the business of writing code, and assuming you want to conduct said business in Python, I would personally start with OpenCV's RANSAC implementation (RANSAC is a common algorithm for aligning images based on keypoints):
https://docs.opencv.org/3.4/d1/de0/tutorial_py_feature_homography.html
Good tutorial here:
https://learnopencv.com/image-alignment-feature-based-using-opencv-c-python/
Even once you have aligned images, background subtraction is a game of heuristics: this is hard even for camera traps, which are literally bolted to a tree. But if your objects are large and salient relative to random other things that change between images (shadows, moving grass, things getting wet, etc.), doable.
Definitely let this Slack know what you learn!
*Thread Reply:* Although I have never tried background subtraction with drone images, my gut feeling is that the precise alignment is going to be very difficult, especially with off the shelf drones. If you have access to high frequency telemetry (self-built drones), it might be of some help. If you are flying off-the-shelf, perhaps anafi is good option to go with. They are cheap and come with very 'accessible' telemetry and programmable control methods -- so you can program paths to fly repeatedly, for example.
also, going back to the original motivation -- would be great to know the actual context/scale of data reduction necessity. Perhaps there a completely alternative solution may emerge.
*Thread Reply:* hi Paul, it does matter what type of wildlife you want to count. Are you looking at elephant and rhinos (as a must), and at lion, cheetah and other (smaller to much smaller) species as a nice to have?
*Thread Reply:* I definitely intend to explore this further based on the responses so far but not putting all my eggs in one basket. The drone I will be using is a custom built fixed-wing with very accurate on-board gps but perhaps even that is not sufficient. Seems like someone has to go out and try 🙂 @Jan Kees I am looking at ‘large’ mammalian herbivores, i.e. impala and bigger
For those interested, here is a (technical) writeup of a project we have done in the Carpathian mountains to try and keep bears 🐻 out of villages and prevent human-wildlife conflicts. In many ways this is similar to human-elephant-conflicts, but there were some distinct (technical) challenges we had to deal with, for example Romania is not as sunny ☀️ as Africa :)
https://engineering.q42.nl/ai-bear-repeller/
*Thread Reply:* Just look at one of the beautiful images our system captures, this one is from this morning ❤️
*Thread Reply:* Awesome work! 😊
*Thread Reply:* this is my go to: https://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=circle®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=0&networkShape=4,2&seed=0.14028&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false|https://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=circle®Dataset=[…]se&problem=classification&initZero=false&hideText=false not sure if the right level though.
*Thread Reply:* I've turned this into a gif in the past: the visualization right at this point in this code.org video is pretty good. You might be looking for something more explanatory of the nuts and bolts, but I feel like it visualizes the idea of model training in a helpful way: https://youtu.be/KHbwOetbmbs?t=126
*Thread Reply:* i don't know if this is the right level/tone: https://www.linkedin.com/posts/bettymohler_post-ugcPost-7018185045085409280-YW_J/
*Thread Reply:* Following, as I similarly have a collaborator organization interested in using it with a UI
*Thread Reply:* I created a GUI to deploy MegaDetector and train custom species detectors: https://github.com/PetervanLunteren/EcoAssist
*Thread Reply:* EcoAssist (which Peter referred to) is the most common GUI for running MegaDetector locally. CamTrap Detector (https://camtrap.net/detector) is also quite polished.
I track all GUIs and other non-traditional ways of running MegaDetector here:
https://github.com/agentmorris/MegaDetector/blob/main/megadetector.md#is-there-a-gui
We also have a 100.000% success rate at helping ecologists get MD running at the command line :), which is sometimes still the best option, depending on the project.
Always feel free to point people who have questions to cameratraps@lila.science (that's me and Siyu), we love hearing from users and can usually quickly route people to the right tools given their specific project requirements.
*Thread Reply:* Thank you Dan! I will point them towards the gui and eventually the option of sending you guys an email, I was hesitant about sending them directly to the latter, didnt want to overload you all.
*Thread Reply:* To my knowledge, no one has trained a Very Big Model on LILA datasets, so there's not a uniform benchmark. A couple notes that may be useful, though:
- I'm increasingly encouraging everyone to think of "all the camera trap data on LILA" as one dataset. It's clearly evolved as a collection of individual datasets, but because it's been mapped to a common taxonomy and because there is a Big Giant .csv file that represents every camera trap image on LILA (both at https://lila.science/taxonomy-mapping-for-camera-trap-data-sets/), you can treat it as one big dataset.
- All of the camera trap data on LILA (except for thermal or stereo data) is also available as a Hugging Face dataset (https://huggingface.co/datasets/society-ethics/lila_camera_traps), based on that same Big Giant .csv file, which also may facilitate benchmarking.
- I have been trying to keep track of papers that report numbers on individual LILA datasets; look for the "LILA" tag here: https://agentmorris.github.io/camera-trap-ml-survey/#papers-with-summaries... and tell me what I'm missing!
*Thread Reply:* Also note that there is no precise location information attached to any data on LILA, though where possible, the Big Giant .csv file indicates country/continent/state.
*Thread Reply:* Sounds good, thanks a lot for these resources. I will reach out to you if I come across any papers apart from those mentioned in your repo.
*Thread Reply:* I have executed three projects, two of which are deployed and running on the field. Please refer to the following research in reverse chronological order:
- Hakunamatata - This was a hackathon on Driven Data where the winning model used a fine-trained EfficientNet model on Serengeti dataset. I believe this was around 2019-20
- DeCaTron - We have deployed a two-stage model (MDv5 + Shifted Window classifier) in one of our projects in India.
- ReWilding Europe - This project was centered on an ambitious initiative to reintroduce wildlife into the forests of Europe. We utilized the Yolov7 backbone to train and subsequently deploy a model targeting 30 distinct species. A primary challenge you may encounter pertains to training for the long-tail distribution, especially in scenarios where data is limited. Additionally, it's crucial to consider how your pipeline addresses and tracks false positives.
Hi all. We have 10 funded PhD positions to offer on "Bioacoustic AI"! See https://bioacousticai.eu/ and click on "Apply for a PhD position" to link directly to the currently-open opportunities
*Thread Reply:* This looks incredible, thanks for sharing! I was wondering if you know of any opportunities like this that are in the US and open to candidates with a PhD already?
*Thread Reply:* Hi Amara - I'm no expert on the US scene I'm afraid, but I'm sure there are others here who can give tips!
*Thread Reply:* Here's a postdoc option at Cornell: https://academicjobsonline.org/ajo/jobs/24693
Hi! On Thursday, @Nico Lang from Copenhagen University is speaking on global vegetation monitoring with probabilistic deep learning in our seminar series RISE Learning Machines. The seminar is open to all on Zoom (and in person in Lund, Sweden), but requires registration! 15:00 CET. https://www.ri.se/en/learningmachinesseminars/nico-lang-global-vegetation-monitoring-with-probabilistic-deep-learning
*Thread Reply:* I don't have the paper, and the email address I have for Hayder is his Missouri email (presumably that's the one that no longer works), but Roland Kays is a co-author on the paper and IIRC one of Hayder's co-advisors, so Roland can probably help. I'll DM you with Roland's email address.
Hey everyone! ☕🏔️📡🧊 One of the AI for Conservation community members, @Nora Gourmelon, published the CaFFe benchmark dataset for tracking glacial calving fronts with SAR imagery. Nora was kind enough to tell me about her research so that I could write a popular science article about it, which was published this morning 🙂
If you're interested glacial calving, computer vision for climate, and supporting your fellow AI for Conservation community members, check out the blog post! https://medium.com/voxel51/caffe-calving-fronts-and-where-to-find-them-1ff57520da45
New NSF Global Center on AI and Biodiversity Change!! Led by @Tanya Berger-Wolf with @Justin Kitzes @David Rolnick Graham Taylor Kaitlin Gaynor Marta Jarzyna Laura Pollock @Oisin Mac Aodha @Devis Tuia @Tilo Burghardt Bernd Meyer and me. Very honored to be a part of this!
*Thread Reply:* This is just amazing news!!! Looking forward to what this initiative brings to our field!
*Thread Reply:* Congrats 💜
Hi everyone 👋
I would like to introduce my recently founded company, Addax Data Science. 🚀 At Addax, we have the simple mission of providing tools that enable ecologists to spend less time on boring tasks and more time on conservation. Think statistics, software, automation, and custom identification models for bird vocalizations or camera trap images. Visit the website for more information: https://addaxdatascience.com/
Do you have any questions or want to know more? Don't hesitate to get in touch!
*Thread Reply:* I've heard reports of several research collaborations getting started at the CVPR bird walk, which is awesome 🎊
*Thread Reply:* I'm going and would love to join!
*Thread Reply:* I’ll be there and I’d love to join in! 🐥
*Thread Reply:* @Devis Tuia as if you would wake up early 😂😂
*Thread Reply:* it’s in my DNA. But I was annoucning that sadly I wont’ be in Paris at all
*Thread Reply:* I’m also going and would be keen to join 🙂
*Thread Reply:* Working with the Pl@ntNet people I should be proposing a plant walk, but I'd be happy to join for the birds 😅
*Thread Reply:* I'll be there and would love to come too!
*Thread Reply:* Local in Paris and would love to join as well for bids and plants :)
This is so cool!! Thought I’d share this article about BirdCast (1) birdCast can help target city wide lights-out campaigns to correspond with high migration time (2) New York city audubon uses it to predict number of dead birds they’re gonna see and plan accordingly (3) birds become “trapped” in the 9/11 tribute lights, and volunteers monitor that and turns it off when too many birds are trapped (4) you can see speed, direction, and altitude of migrations!! (5) it can warn farmers when to be on high alert for avian flu
/https://tf-cmsv2-smithsonianmag-media.s3.amazonaws.com/filer_public/a8/9f/a89f0231-a922-473f-a8f4-6133d0be3b85/2023_birdcast_forecast_screenshot_web.jpg)
Hi everyone, I have a question regarding the MegaDetector detections available for lila.science. Some datasets (e.g. Orinoquía Camera Traps) have multiple animals annotated per image in some cases. Corresponding to these, the MegaDetector also provides multiple detections with high confidence. Is there a way to may the detections to respective categories? The MegaDetector categories are just animal, person, and vehicle, so they are not useful in this case.
*Thread Reply:* There's not a way to do this reliably, sorry. If you are, for example, combining those MegaDetector boxes with image-level labels to train a multiclass detector, you will likely have to either (a) ignore images with multiple species or (b) manually assign labels to boxes for those images. FWIW, outside of the African datasets, multi-species images are extremely rare on LILA, so you probably don't lose a lot of information if you discard multi-species images in, for example, Orinoquia Camera Traps, and/or it would be super-duper fast to manually assign labels for the relatively small number of boxes this affects.
If you are working with the Snapshot Safari data (where multi-species images are common enough to possibly want a solution), you could run a classifier trained on cropped animals on each box in each of those images, and I think 99.9% of the time, even if the classifier isn't great, you would be able to assign boxes to labels (e.g. if you had zebras and warthogs in the same image, you only really need to rely on the classifier to assign a higher probability to zebra than warthog or vice-versa to help you assign those boxes). But even there, I think it would be faster to do it manually.
*Thread Reply:* Actually if you are training a multiclass detector, the most elegant solution is probably to leave those out, train an initial detector, use that detector to assign labels to that small set of boxes, manually clean up what should now be a very small number of mistakes, and re-train.
*Thread Reply:* Sounds good, thanks for the information!
Amazing seminar last Thursday with @Nico Lang in our Learning Machines series. Thanks @Sara Beery, @Ben Weinstein, @Alexander MathisEPFL and others for attending in-person and contributing to a great event.
We have created a new youtube playlist with all our environment-related talks. Check it out!
https://www.youtube.com/playlist?list=PLqLiVcF3GKy0-jZFGg-VqLzh51LqCfduN
November 2, Pria Donti is visiting our seminar! More info: https://ri.se/lm-sem
*Thread Reply:* Sorry, because I’m on my phone I can’t edit your google docs file. I thought this book was an interesting read: AI in the Wild - Sustainability in the Age of Artificial Intelligence - By Peter Dauvergne
*Thread Reply:* Thanks! Feel free to add here and I'll move them to the sheet.
*Thread Reply:* We recently released a review on AI for elephant monitoring https://arxiv.org/abs/2306.13803
Hi, I am unable to download the annotations for Missouri camera traps in LILA https://lila.science/datasets/missouricameratraps. The annotations link hosted by LILA is non-functional https://lilaannex.blob.core.windows.net/lila-annex/missouri_camera_traps_set1.zip Any help is appreciated.
*Thread Reply:* Good catch! Should be fixed now (new link). The issue was... no, actually the issue is really boring, but it should have only impacted this particular file. Let me know if anything looks off, the metadata for this dataset has been... complicated (see the note on the page about some bounding boxes that aren't quite right).
*Thread Reply:* The page says that bounding boxes provided by the authors of Missouri camera traps dataset aren't accurate. How about the megadetector results available on LILA? Are those reliable to be used instead?
*Thread Reply:* It's not that the bounding boxes aren't accurate, rather that for 79 images (there is a list on the page), only one bounding box is available, when in fact there are multiple animals in the image. So if you are using the dataset to train a detector, I would leave out those 79 images. If you are cropping the boxes and using those crops to train a classifier, they're all fine.
But yes, I also expect that the MDv5 results for this dataset are good too.
*Thread Reply:* Hi, while processing the LILA datasets, I found that for most datasets, the annotation images and the images available for download correspond to slightly different sets of images. Just want to make sure there is no processing issue at my end.
*Thread Reply:* The only difference you should see is that images with humans are in the metadata, but not in the actual image folders. That would also include images labeled, e.g. "vehicle", and in some cases "domestic dog" or even "horse". When posting datasets to LILA, we avoid posting images that even might have people in them.
If you see a significant number of images in the metadata that are not labeled human and are not present in the cloud bucket(s) or zipfile(s), like missing images of lions or bobcats or whatever, please let me know at info@lila.science. Thanks for checking everything carefully!
*Thread Reply:* Hi Drea, this is really interesting and something ive been trying to research lately. Can you point me at any papers discussing the extend to which indicator species can proxy broader biodiversity and ecosystem health?
Cool use of camera-traps for non-wildlife insights! https://x.com/jamiealison30/status/1706953803474387324?s=20
*Thread Reply:* I love this!! I am catching up on all my missed slacks and this looks awesome. Thank you for sharing Carly 🙂 !! And for tagging me Justin 🙂
*Thread Reply:* we could give YOLOv8 a try on this... just need a bit of time (after IROS this week and our next field trip.. we are back in the office on 24th Oct)
*Thread Reply:* SG, drop me an email when you're back (agentmorris@gmail.com), and I'll send some more information.
*Thread Reply:* Perhaps https://github.com/robot-perception-group/smarter-labelme?search=1 could be of help.. we built it on top of Labelme. We further adapted it for behavior annotations here https://github.com/robot-perception-group/animal-behaviour-inference (perhaps that thread you meant was about out smarterlabelme tool when we released it). A short summary of what we do in this tool is tracking (predict-detect loop like (but not exactly) a Kalman filter) the bounding boxes. Both predicter and detector are DNNs. A human in the loop fixes small deviations if any. We see almost 10-fold decrease in time spent annotating/correcting.
The company I work for, Impact Observatory, has started offering the ability to more easily download our free annual land use and land cover maps clipped to any area of interest for any year from 2017-2022.
Our global maps were already open access, but now you can order them through our store and get your desired data emailed to you already clipped to your area of interest along with various metrics about your data. Thought this might be helpful or interesting to some folks in this group! https://x.com/ImpactObserv/status/1702684441271804004?s=20
*Thread Reply:* I just placed a test order for a small area around where I live to try out the tool; the process was very slick!
*Thread Reply:* Glad to hear it @Dan Morris!
*Thread Reply:* Cool product! Is there some way of saving the polygon or getting multiple years in one order?
*Thread Reply:* Thanks, @Paul Allin! Using the same custom polygon for multiple orders is a much-requested feature, and one that we definitely want to offer soon!
🚨 Last call for submissions to the 2023 NeurIPS Workshop on Computational Sustainability! https://www.compsust.net/compsust-2023/
- Submission Deadline: Oct 3, 2023 (AOE)
- Notification of Acceptance: Oct 21, 2023
- Workshop: December 15th, 2023; New Orleans, Louisiana
We hope to see some submissions from the AI for Conservation community!
Feel free to message me or contact neurips-workshop2023@compsust.net with any questions.
}
*Thread Reply:* The negative results focus is a great idea. Negative ML results in the literal sense of low accuracy are often a little tricky because it can be impossible to say whether the results were fundamental or just related to implementation issues. But negative results where the ML worked (in the sense of high accuracy), but the model still didn't end up being useful (because the test domain surprised you, or you built a model you thought the user wanted but it wasn't actually what they wanted, or the user thought 91% accuracy was good enough but it turns out they needed 94% accuracy, or whatever)... those are super-interesting! Excited to see what submissions you get in the "negative results" department.
FruitPunch AI for Turtles Challenge coming up! 🌊🐢 Dive into the world of Sea Turtle Conservation! 🐢🌊
We're thrilled to announce the 𝐀𝐈 𝐟𝐨𝐫 𝐓𝐮𝐫𝐭𝐥𝐞 𝐂𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐞! 🐢🤖
Our goal is to develop cutting-edge computer vision software that can recognize and distinguish individual turtles through automated identification. Current turtle monitoring methods are intrusive, time-consuming, and expensive, posing challenges to organizations dedicated to protecting these endangered creatures.
How it Works: Sea turtles have unique facial scales, much like human fingerprints. AI can identify these distinct characteristics, helping researchers associate them with individual turtles. In this Challenge, we partner up with @ Sea Turtle Conservation Bonaire, an amazing organization committed to protecting these precious animals and their habitat.
In this challenge, we aim to: • Explore Siamese and Triplet networks for turtle identification. • Investigate state-of-the-art techniques like 𝐀𝐫𝐜𝐅𝐚𝐜𝐞. • Develop data pipelines for processing training and testing sets. • Create an easy-to-use GUI in collaboration with researchers for efficient turtle identification. We're seeking passionate individuals from various backgrounds, including AI/ML enthusiasts, software developers, biologists/conservationists interested in AI, and computer scientists. We expect a commitment of ~8 hours per week. You will be joining weekly meetings, masterclasses, and presentations.
Join us in making a real impact on turtle conservation in Bonaire and contribute to preserving these incredible creatures! 🌊🐢
Learn more about the challenge on our website: 👉 https://www.fruitpunch.ai/challenges/ai-for-turtles
*Thread Reply:* Do the datasets for these challenges get published somewhere afterwards? I couldn't find a link to data on past challenges on the website.
*Thread Reply:* Have you chatted with the ML engineers at WildMe yet? @Jason Holmberg (Wild Me) and his team may have some good insights on this challenge 🙃
*Thread Reply:* Hi Elizabeth, it depends on the Challenge Owner who provide the data if we can publish their data. If we can we always publish it on our platform but for many cases we sadly cannot. In this case we have 3 datasets, two of those are from the Caribbeans, they will only be made available to challenge participants for now, the last dataset is a public data set from South Africa.
*Thread Reply:* @Katelyn Morrison we have most definitely been in touch, mostly regarding our animal classification cases, not yet on an identification case, though I'd be eager to collaborate. I see they already have a Internet of Turtles project
*Thread Reply:* @Jason Holmberg (Wild Me) I read you use a SIFT based method. I'd be interested to hear about your experiences with this algorithm!
*Thread Reply:* Hi Sako, what kind of annotations does the public dataset from South Africa contain? Can you post a link to this public dataset, please? Thanks, Urs
*Thread Reply:* Hi Urs, You can find the data and the explanation of the labels here: https://zindi.africa/competitions/turtle-recall-conservation-challenge/data
*Thread Reply:* Hey @Sako Arts, I've been approached to advise a sea turtle citizen science startup on AI methods for individual ID of sea turtles. Wondering if you have any suggestions on open source models/techniques I should be looking at? Cheers 🙂
*Thread Reply:* @Casey Clifton ArcFace loss + EfficientNet is our state of the art for general re-ID. Turtles are on our roadmap (data from iot.wildbook.org), but it might be Q1 2024 before we have time to train.
*Thread Reply:* Thanks @Jason Holmberg (Wild Me)!
*Thread Reply:* @Timm Haucke will be representing our group there!
*Thread Reply:* Yes, you have to have an alternative option like non-lion. Otherwise your attempting to train lion vs NULL. That’s the easiest route at least.
*Thread Reply:* Is the idea to classify crops from megadetector or to train a separate lion-only detector? If it's the second, and you have bbox labels, then it should use the background not in the lion boxes as the negative class.
*Thread Reply:* Great 🙂 Yes, it will classify crops from megadetector. I'll go and find some non-lion crops to add a second class!
Different scenario: if I wanted to train a classifier for lion and zebra , do you recommend to add a third class other? Or could I in this case use the confidence values to decide which animals are not lion nor zebra ? For example, if the crops from megadetector don't get a classification for either lion or zebra above the threshold value X, then it'll be a classified as other.
*Thread Reply:* This is one of those questions where I think you get a free PhD in computer science if you can provide a universal answer. :) I'm assuming you have labels for a bunch of other species too, e.g. you're working with Snapshot Safari data, or more generally, it's likely that whatever your training data is, it's not only lion and zebra that were labeled. Even if you're only interested in lions and zebras, I'm 71% confident that you want to include labels for at least the other common classes as well, but you probably don't want to use so many training examples that they overwhelm lions or zebras. E.g. if you're in an ecosystem where wildebeest and elephants are also common, you could (a) lump them into an "other" class or (b) give them their own classes, and maybe lump the less common animals into "other". I'm 71% confident your performance on lions and zebras will go up if you choose (b), but I'm not sure. And it's hard to say how far down the list of classes you want to go before you start lumping things into "other". If you have 10 species that are as common as lions, I think you want them all to be their own classes, but I'm not sure. Let us know what you learn!
*Thread Reply:* ❤️ I really like this answer.
*Thread Reply:* I was hoping for one of those answers with which you'll get a free PhD... 😁 Thanks guys! That makes a lot of sense 🚀
*Thread Reply:* Here's the scene ID https://planetarycomputer.microsoft.com/api/stac/v1/collections/sentinel-2-l2a/items/S2A_MSIL2A_20231011T025631_R032_T49NDJ_20231011T101112
*Thread Reply:* And a link to look around on the PC https://planetarycomputer.microsoft.com/explore?c=110.6857%2C7.9889&z=9.99&v=2&d=sentinel-2-l2a&m=Most+recent+%28any+cloud+cover%29&r=Natural+color&s=false%3A%3A100%3A%3Atrue&sr=desc&ae=0|https://planetarycomputer.microsoft.com/explore?c=110.6857%2C7.9889&z=9.99&v=2&d=sentinel-2-[…]Natural+color&s=false%3A%3A100%3A%3Atrue&sr=desc&ae=0
*Thread Reply:* there is separation there as well
*Thread Reply:* 1000% agree, but I can't take any credit for it:)
*Thread Reply:* yeah its super nice — responsive, fast, does exactly what you think it should do with zero need to read documentation
*Thread Reply:* That looks like the combination of specular reflections and different acquisition times / angles. Some additional info: https://gis.stackexchange.com/questions/332634/water-reflectance-effect-on-sentinel-2-images
*Thread Reply:* wow. That makes total sense. Thank you for responding.
*Thread Reply:* @Nathan Jacobs your explanation makes sense but also I would expect to see this effect more often. To date I have only seen it once, and we look at a significant amount of S2 TCI imagery.
*Thread Reply:* maybe it's a scale issue... the reflections need to be large enough to not get averaged out w/in a single pixel... honestly not sure
*Thread Reply:* I remember back in the days Quickbird had similar isues because the IR band was not acquired simulataneously to the RGB. SO you could see moving objects by making a false color composite
*Thread Reply:* Appreciate all the responses. For the moment we are just going to tell our users that this is a known anomaly with the cause likely attributed to reflectance as Nathan mentioned. If we see this more often, and if it has an impact on our precision, then we will revisit and remove this effect — perhaps via a conventional noise filtering technique (erosion, dilation).
Hi everyone, I’m writing to share some tool-building work @Maddie Cusimano and I did which might be useful for other folks working with long audio recordings: https://github.com/earthspecies/voxaboxen.
This is a framework to train a sound event detection model directly on audio plus Raven selection tables. It differs from most other sound event detection approaches that we know, in that it uses an object detection framework (similar to YOLO) to box events, rather than a segmentation or classification framework. This approach is useful for labeling data at a fine temporal scale, which we wanted for looking at interactive vocal behavior of animals.
We talk more about our approach in this blog post: https://www.earthspecies.org/blog/voxaboxen-new-tool-to-support-annotation-of-large-audio-files. Happy to answer any questions, hear comments, and receive feedback! We can also help you to run this on your data.
.jpg)
*Thread Reply:* How many other people immediately went to YouTube after reading this post and searched for "what sound does a meerkat make?". I did. It was worth it. Meerkats just got more adorable, which is saying something.
*Thread Reply:* Great work! I really enjoyed reading through the article 👍😊
Hi all, I am writing to share several remote sensing computer vision models that we recently created and open sourced. • https://github.com/allenai/vessel-detection-sentinels • https://github.com/allenai/vessel-detection-viirs These models support real time streaming computer vision (vessel detection) from NASA’s and ESA’s publicly available satellite imagery (VIIRS, Sentinel-1, and Sentinel-2) in our platform (Skylight: https://www.skylight.global/). Many of these models were built by or in close collaboration with Prior (AI2's CV division, and especially Favyen Bastani who is incredibly talented at geospatial AI). Our team has benefitted massively from all of the expertise and insights from users in this community and we are grateful for feedback about these models (engineering, modeling, documentation, or otherwise).
*Thread Reply:* This looks awesome! Our group has some projects trying to understand how vessel traffic noise affects marine mammal presence, but we are only using AIS data. We're always concerned about how many vessels we might miss because they don't have AIS
*Thread Reply:* Echoing the congrats. 🌏 We have used VIIRS to detect bioluminescent milky seas, and are working toward ML methods to automatically flag them in the data streams. Your project looks like a helpful launch point. https://www.nature.com/articles/s41598-021-94823-z
*Thread Reply:* Very cool project! (and brings me back to swimming in bioluminescent bays in Puerto Rico with my wife.
FWIW There are downloading scripts (for creating datasets) NASA in there as well you may find useful. We use this service in a real time platform, and we actively maintain/monitor this code — if you notice any issues feel free to open an issue, and if you want any more details we would be happy to talk — we think VIIRS can be a very powerful technology, and there is even a third satellite that was recently launched (NOAA-21) that will enable us to have 3 passes globally every night. (I gave a talk on this model recently at the NOAA-AI conference: https://noaaai2023.sched.com/event/1SA4O/live-demo-shedding-light-on-shadowed-waters-geospatial-computer-visions-role-in-tackling-illegal-fishing)
<a href="https://github.com/allenai/vessel-detection-viirs">allenai/vessel-detection-viirs</a>
*Thread Reply:* Hey Sarah, myself and some prior colleagues spent a lot of time developing a workflow to rename, blur, and bin photos using MegaDetector, and I have recently been delving into training regionally-specific classifiers for Washington, British Columbia, Alberta (though only broad classification into carnivore vs. non-carnivore at the moment). Not sure if I can be of much help, but I’d be happy to chat if you have any specific questions!
*Thread Reply:* Hi Sarah (also hi Michael!), I am working on similar projects in Oregon and would be happy to chat as well
*Thread Reply:* Yes to both of you! I'll reach out to you each to schedule a shortish call. thanks for responding
*Thread Reply:* WildMe.org here in Oregon. Please include me in your discussions. We take imagery to the individual ID level where possible and would love to work with regional states on AI for aerial surveys too
*Thread Reply:* Hi all! I am in Oregon as well and would love to join the conversation. I am part of an international group focused on automating insect monitoring. Some regional support would be very welcome 🙂
*Thread Reply:* @Cara Appel I will be at OSU this weekend for the PNW Lepidopterists Workshop. They are going to show off a new collection in the newly remodeled Cordley Hall! https://osac.oregonstate.edu/events/44th-annual-pacific-northwest-lepidopterists-workshop
*Thread Reply:* Hi Sarah, Our workflow here in Toronto uses megadetector and timelapse too!
*Thread Reply:* @Michael Bunsen Sounds so interesting! I will be out of town. Do you know Michael Getz at OSU? Looks like he's not on this channel, but yet another Michael in the AI world and he does pollinator work.
*Thread Reply:* @Cara Appel I don't know that Michael but I am intrigued! I appears that he is a leader in the field of truffle dog training as well 😃
*Thread Reply:* Hi Sarah - I have worked on creating a workflow and species level classification for Wildling Europe and another project in Telangana, India. In India we are working with local state governments, to setup their machine annotation pipelines for the millions of data points that they gather through regular monitoring. Happy to have a chat and discuss any potential collaborations.
*Thread Reply:* Hi Sarah, We do something similar in SF Bay Area with Felidae Conservation Fund Wildepod Project (https://felidaefund.org/projects/community/wilde-pod) We have about 1.6 million images (and counting) collected from a couple of hundred camera traps scattered around bay area. A small team of volunteer engineers (including me and @Abhay) built the wildepod.org site which allows the field volunteers to upload the image sets, which are then processed by MegaDetector and then annotated for species etc by a group of other volunteers through a custom interface. It's a Django site hosted on App Engine and I'm happy to share details with you based on your interest.
We've been meaning to implement automated species classification as well but so far this year keeping abreast of front-end features and site breakages etc. Hopefully early next year!
Would love to collaborate with folks doing similar work! 🙂
*Thread Reply:* Thats really neat Prabath - It sounds like what so many of us are wishing for -- I will take a look!
*Thread Reply:* I am not sure whether it can help for a complete workflow but we have a North American species detection model that we are trying to expand to more species and into result dashboards that are useful for users. https://www.conservationai.co.uk/ I am happy to discuss if this can be part of a workflow that is of use for you and others. We are also always looking for more images for species in North America to train the model to include more species of mammal.
*Thread Reply:* I am attending the conference Monday. I am in the symposium "Agent-based modeling for wildlife management". There is a get-together for the symposium planned afterwards at Down One Bourbon Bar Monday, Nov. 6th, at 6pm
*Thread Reply:* Awesome, thanks @Toryn Schafer!
*Thread Reply:* I'm here as well, come by the WEST booth to chat!
*Thread Reply:* Amazing talk. Intrigued by the possibilities that optimization in the loop machine learning can give!
*Thread Reply:* @Elizabeth Bondi-Kelly has a dataset for aerial detection in thermal data
*Thread Reply:* Hi Ben, we have some experience in this context through our AirCap project. I will be glad to connect. you can check the project, pubs and dataset here : https://www.aamirahmad.de/projects/aircap/
*Thread Reply:* ... and all open source code here https://github.com/robot-perception-group/AirCap
*Thread Reply:* Thanks for tagging me @Sara Beery! Yes, here is the dataset: https://sites.google.com/view/elizabethbondi/dataset?authuser=0. I also think that the ethical discussion is incredibly important, and I'd like to refer to some great work that I've seen in this space - this is of course not exhaustive, and much has come from Wildlabs!
• https://conbio.onlinelibrary.wiley.com/doi/10.1111/csp2.374 • https://besjournals.onlinelibrary.wiley.com/doi/10.1002/2688-8319.12033 • https://www.youtube.com/watch?v=Ge6_EF0z83U • https://fnigc.ca/ocap-training/ • https://news.usask.ca/articles/colleges/2020/engaging-northern-expertise-strengthens-ecological-science.php • https://link.springer.com/article/10.1007/s13280-015-0714-0 • https://vimeo.com/771752163
*Thread Reply:* I’m currently working on something that relies on the Sentinel API, the goal being to DL/process imagery daily. I’ve found the API extremely clunky but have a pipeline that I believe works. I’m very much trying to avoid going with a downloader-pays model, which seem to be getting more common… Interested in getting your perspective here and happy to chat more.
*Thread Reply:* We are having quite a bit of friction with this and we are happy to pay but its not obvious if that would solve our problems. Its not trivial to support these NRT use cases, and requires significant resources to do so, and then there is the question of at what SLA. There are groups moving to the cloud/s3 (like NOAA for some of their products), which enables a more modern AWS API.
Our use case is rather demanding — we support organizations world wide and they need to know as soon as possible whether there are vessels somewhere/doing something they aren’t supposed to be doing). Our computer vision is actually quite fast in comparison to the downlink/transfer from ESA’s servers.
*Thread Reply:* | Its not trivial to support these NRT use cases, not us — I mean for NASA/ESA etc.
*Thread Reply:* FWIW We have had great experience with NRT NASA data (we use SuomiNPP/NOAA20, and soon NOAA21 satellites) — there servers are basically always on -- and we only see performance degradation when there is an actual on board satellite issue causing data corruption/preventing downlink to earth. We
*Thread Reply:* NASA’s NRT data is free — which is kind of surreal when you think about how serious/big of a lift supporting that volume of data and NRT on top of that
*Thread Reply:* Our goal is to open source the best solutions we find so that groups don’t have to reinvent the wheel every time. Here is the service for NRT processing of Suomi-NPP and NOAA-20 — I linked to it earlier in this channel, but didn’t underscore the downloading component, in case you find the code useful: https://github.com/allenai/vessel-detection-viirs and happy to chat of course as well
*Thread Reply:* Thanks for sharing the repo and for your take on this! Nice to know about the reliability of the Suomi/NOAA NRT data. From my memory Sentinel data is available within about 4 hours of acquisition (when DLing directly from the ESA Sentinel API) which I could see would not be ideal for your use case. I’m more concerned with satellite revisit time then time to data availability (once it’s in the hours range). Happy to share the code I have for Sentinel-2 once it’s cleaned up/fully operational.
*Thread Reply:* Hi we (Skylight) are exploring geospatial tuned LLMs (bespoke to Skylight) to remove barriers for entry/accelerate adoption/help growth etc — we built (and deployed) one for AI2's hackathon with AI2's Aristo team. I went to CMU and this caught my eye 🙂. Feel free to reach out to my email directly: patrickb@allenai.org Skylight is a marine conservation platform.
*Thread Reply:* https://www.skylight.global/
*Thread Reply:* Hi Patrick! Thanks for the response, this sounds like a really interesting project. We’ll follow up over email to chat more.
Job opportunity at the Turing Institute on Autonomous Systems for Biodiversity Monitoring. Help develop algorithms for adaptive sampling on autonomous underwater vehicles!
*Thread Reply:* Sure. jason@wildme.org please.
*Thread Reply:* I too would love to have a look.
*Thread Reply:* I created a Bluesky feed on AI for climate change! https://bsky.app/profile/did:plc:fyw3wpo7owjjc362rp42upzo/feed/aaabbqsgt6bd2
*Thread Reply:* @Jason Holmberg (Wild Me), @Graham Wallington, did you register? I cannot find you at Bluesky.
*Thread Reply:* I don't know if this counts as "not covered by LILA", but on the "other datasets" page on LILA, I list a few other camera trap datasets in the section called "Terrestrial wild animal images (ground-based sensors)":
https://lila.science/otherdatasets#images-terrestrial-animals-ground
*Thread Reply:* I haven't seen specific programs, but I have seen people apply to either CS-related programs or another domain (i.e., civil engineering or engineering and public policy) and then work with/get advised by professors who work on CompSust related topics. There are also lots of summer schools like CV4Ecology summer school and the Climate Change AI summer school. 🙂
Looking for people to follow on Bluesky! Looks like a promising alternative to Twitter. Here is my profile: https://bsky.app/profile/olofmogren.bsky.social
*Thread Reply:* I still have a spare code if anyone wants to sign up!
*Thread Reply:* I have now created a public feed for AI for climate change! https://bsky.app/profile/did:plc:fyw3wpo7owjjc362rp42upzo/feed/aaabbqsgt6bd2
*Thread Reply:* Apparently there's a 'bear guy' in alaska with a huge number of images, but he's pretty off the grid. A friend was trying to contact him recently - ill check and see if he got through!
*Thread Reply:* Oeh sounds exciting, would be great to get in touch!
*Thread Reply:* Any of the folks at NatureMetrics or SimplexDNA would be great for eDNA advice!
*Thread Reply:* My team at WWF has done some comparison studies on species captured, costs, etc. between the two methods - happy to connect you if it's of interest! https://www.nature.com/articles/s41598-021-90598-5
*Thread Reply:* Thanks @Talia Speaker! I'll read this and come back to you
*Thread Reply:* Mark Davis over at Illinois Natural History Survey is doing some interesting work with eDNA, just saw a presentation from him comparing the species found with eDNA vs camera traps (surprisingly little overlap!)
*Thread Reply:* That's interesting @Riley Knoedler - do you have a link to the pres i could see?
*Thread Reply:* @Casey Clifton It was at the REWI Solar Symposium conference so I don't have a way to distribute the presentation slides, but you might be able to discuss it with Mark! I'm not sure if that work is published yet.
*Thread Reply:* We have just finished the data collection for comparison of eDNA methods (air, dust, and water ) to camera trap data. Happy to chat
*Thread Reply:* here's our paper on iDNA and camera traps https://onlinelibrary.wiley.com/doi/full/10.1002/edn3.46 Biggest limitation in our mind is how to go beyond species detection and into population estimates.
*Thread Reply:* Oh hey I’m at CUNY too!! I’m a PhD student at the GC (Biological Anthro dept). Based out of Hunter though.
*Thread Reply:* For your question - there are lots of good resources on WILDLABS about different trail cams, and it might also be a good place to re-post your question as well.
*Thread Reply:* What are you currently using/
*Thread Reply:* Hi Filippo, We are working on using AI for fish monitoring. --> streamocean.io Let me know if you have some questions
*Thread Reply:* Here is the recording of this excellent talk! https://www.youtube.com/watch?v=7yXsFFbWgcs&list=PLqLiVcF3GKy0-jZFGg-VqLzh51LqCfduN&index=1
just a reminder for this beautiful position for Deep SDMs in my team! If you feel like your next life step could be in Switzerland, please apply! We will be starting looking at the application in 10 days!
Pleasewrite me a msg if you have questions, happy to give info!
}
*Thread Reply:* Hi, thanks for sharing this! I have a few questions, will send them in private chat.
*Thread Reply:* Hi Fadel, I think the AIDE platform could help you out (and sorry everyone if I bring that one up often, it’s really not for bragging). With AIDE you can start labeling your images and use a pre-trained model tuning on those. Then the trained model will propose you candidate detections to review within an active learning loop. You can customize species as you go.. You can contact @Benjamin Kellenberger, who is the mastermind behind it.
*Thread Reply:* Thank you @Devis Tuia, I will look into it 😁
*Thread Reply:* Hi Fadel, you may be interested in a tool that my supervisor Derek Young at UC Davis developed for scripting runs of the Agisoft Metashape photogrammetry software. He also has some work assessing the impact of flight parameters on tree detection using geometric methods: https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13860.
*Thread Reply:* Hi @Fadel, we're building our second round aerial flight detector now for the KAZA Transfrontier survey. Happy to share ideas. @Lasha Otarashvili is the primary on the training effort.
Hi Casey, here are some tutorials:
- https://www.kaggle.com/code/agentmorris/fine-tuning-megadetector
- https://pub.towardsai.net/train-and-deploy-custom-object-detection-models-without-a-single-line-of-code-a65e58b57b03 The second one is more for adding custom labels (i.e., species identification), but can also be used for fine-tuning the existing labels (animal, person, verhicle).
*Thread Reply:* Casey, are you trying to fine-tune MegaDetector in the sense of making it work better on species or camera angles it doesn't work well on, or in the sense of adding a species classification stage? Just a reminder that if you're referring to the latter, you may find it easier not to fine-tune MegaDetector at all, and instead train a classifier on the crops that come from MegaDetector. It's not obvious one way or the other, but if I had to place my bets on one approach, I would place my bets on the two-stage approach, and I think Peter's conclusion has been the same (Peter, don't let me put words in your mouth, but I vaguely remember you saying that).
If it's the former - making MD work on things it kind of works on, but not as well as you'd like - fine-tuning is a good solution. I've just done this for the first time recently (for a couple of reptile-heavy use cases), and haven't written up a tutorial yet, but can provide some tips and tricks if this is your scenario.
Somewhere in between would be the case where MD works well on your animals, but you want to reduce false positives for a particular ecosystem or set of cameras; I don't know that anyone has looked into this, but I think I would recommend a post-hoc classifier in this case (junk/not-junk) rather than fine-tuning. But I would only bother doing this if you have an egregious number of false positives.
*Thread Reply:* Yes, absolutely @Dan Morris. I've tried fine-tuning MD in the sense of adding a species classification stage, but found that that just costs lots of training data, processing power and yielded mediocre results. If you want species classification, I would definitely work with a custom classifier on top of MD. But if it is for improving detection rate (of the original classes animal, vehicle, and person) on "images taken in a specific region", it might be a good approach - but I haven't tried it out myself 😉
*Thread Reply:* Hey @Dan Morris thanks for the info! I'm looking at the former. Specifically, using MD in an Australian setting, where im finding it misclassifies native plants as animals, and also just doesn't recognize some animals - there seem to be more false negatives than other papers have reported. Would love to hear of any tips or tricks you have for this.
*Thread Reply:* Gotcha. We may quickly get to the point where it's faster to do this by email (especially if we need to look at sample images), but I'll take a shot here, and if we move to email, I solemnly swear that if I don't report back to this Slack with whatever we learn, I owe everyone here a granola bar.
But, next questions:
- Confirm that you're using MDv5?
- Are you running MD via rundetectorbatch.py, via EcoAssist, or via something else?
- What confidence threshold are you using?
- Are the misses** reptiles, mammals, birds, or some other scary thing you have in Australia that we don't even have an American word for? Like poison-dragon-spiders, or poison-spider-dragons.
- Are the misses** on things that are really obvious to your eye, or are they tail-sticking-into-the-corner kind of misses?
- Very rough ballpark: about what percentage of your images are actually empty?
- Very rough ballpark: about what percentage of the animals are being missed**?
- Very rough ballpark: about what percentage of MD's positives** are false positives? For (6)/(7)/(8), I know the real answer is "it varies a lot across cameras", but just try to average overall all your cameras.
**All those asterisks are there as a reminder that "positive", "negative", "miss", etc. are just placeholder words for now, they're undefined in this thread until we know the answers to (1)/(2)/(3).
*Thread Reply:* Happy to email / share pics / supply granola
- Yep v5
- Via rundetectorbatch.py
- I've tried a few levels and if i just score a classification of animal vs not animal i get the following precision/recall values a. Conf 0.2 = 0.74 / 0.76 b. Conf 0.5 = 0.91 / 0.7 c. Conf 0.8 = 0.97 / 0.53
- Most of them are night shots of Quenda - the species we're investigating that is native to Australia. As far as i know they're not poisonous, but it's about the only thing that hasn't bitten us yet so who knows...
- They are usually obvious when you look at multiple images in a row (we take 5 in 5 seconds for each trigger) because the animal is the only thing that's moving, but they're basically impossible to detect from a single shot with no context. a. Has anyone looked into detection with temporal context to solve this? I think it'd be essential because Quenda are nocturnal, small, and fast, so they can be pretty blurry, easily hide behind a small bush, etc.
- About 85% of the ones we've labelled so far are actually empty, but i know there are sites we havent labelled yet that are 99% empty
- As per recall numbers above, about 25-45%
- False positives were high but since removing repeating bounding box locations a lot of the FPs due to plants are removed so this is negligible at the moment
*Thread Reply:* Super-helpful. And also I just learned what a quenda is.
At the end of the day, if the thing you're looking for is only visible in image sequences, MD can't see it. So I think there are two questions here: (1) what are your possible paths forward for finding things that are only visible in multiple images? and (2) how can you get the most out of MD? I'll take a shot at (1) here, though hopefully others will chime in too, since I have no particular expertise there, then I'll address (2) in a separate reply.
Re: moving beyond single images...
AFAIK, there is no off-the-shelf tool that will help you find animals that are mostly only visible in sequences. There's Sara's work on Context-RCNN:
https://arxiv.org/abs/1912.03538 https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/context_rcnn.md
...but I would say that's solving a lot more than just the problem of finding a quenda scurrying in the bushes, and is therefore justifiably pretty complex, and I don't know of anyone that's deployed it.
If you are willing to get into the business of training custom models but don't quite want to leave the universe of RGB images (it's hard to overstate how much more complicated the world gets when you leave the world of RGB images, you lose 10 years of computer vision infrastructure development built around RGB images and video), I was peripherally involved with a paper that proposed a bunch of relatively simple ways to fit sequence information into RGB images... things like "swap out the blue channel and replace it with a motion channel":
https://arxiv.org/abs/2005.00116
I'm still a fan of that approach, and if you have enough data, it's relatively easy to explore (since you're still training RGB models). But again, AFAIK no one has deployed this. I think it would be particularly neat to train a detector using one of these approaches, which was outside the scope of that paper (which was focused on classifiers), but you would need to get a whole bunch of boxes, and then do this complicated research-y thing.
And although this might not help you now, if you can capture and label a bunch of videos, you might try training a custom model via Zamba Cloud:
*Thread Reply:* Re: getting the most out of MD...
FWIW, even for easy datasets, I never use a confidence threshold higher than 0.2 with MDv5, and for difficult datasets, I'll go as low as 0.05. So I think you have some room to reduce your confidence threshold, though I don't think it will immediately get your recall up from 0.76 to something super-respectable, because invisible is still invisible. But if 85% of your images are blank, IMO you have a lot of room to compromise on precision and still have AI save you lots of time, if you can get to adequate recall.
But the main tricks I use for difficult datasets are, from most to least important:
- Repeat detection elimination... good for maybe a 2-3% precision bump at reasonable thresholds. For easy datasets, I typically spend about 5 minutes per million images on this, maybe for difficult datasets, this goes up to 10 minutes per million images. Once you get the hang of it, this will always increase precision with no impact on recall. This is particularly important for cases where you want to user super-low confidence thresholds.
- Test-time augmentation, by doing inference via this script instead of rundetectorbatch. This will raise the confidence of everything a little, but it will particularly raise the confidence of low-confidence animals. I.e., when difficult animals are present, there is real signal there, it's not just raising the confidence of every detection. In principle this can improve both precision and recall, but in practice, I've only ever seen it at least slightly hurt precision, while always either helping recall or leaving recall the same.
- Combining results from MDv5a and MDv5b, using this script. Your case sounds like a difficult one that may be out of reach of one-image-at-a-time ML, but I will DM you to see what we can squeeze out of MD.
*Thread Reply:* This is great, thanks. Will reply in DMs!
*Thread Reply:* For questions on the awards, reach out to our new program manager @Adrien Pajot 🙂
*Thread Reply:* I think you should raise this with the editors. The article should be amended with an erratum, since it deviates from the article that was accepted (whether by "accident" or not). The editors should be willing to handle this.
*Thread Reply:* ya, update here, editors said they are looking into it.
*Thread Reply:* Thats great, I've been hoping someone would do this and update. I still get alot of downloads of https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13011, which is getting pretty old at this point. Can I start pointing users towards you? What kinds of models are in there to start?
*Thread Reply:* That looks really cool! Its currently just a wrapper around YOLOv5, so in theory it will work with any YOLOv5 weights. I am using it with a collaborator from University of Michigan with MegaDetector_v5 to detect simple presence/absence of animals in a giant video collection but it should work with other weights (species classifier, etc.) as well with minimal changes.
*Thread Reply:* I have one super-important question: did you make that ASCII art tiger?
*Thread Reply:* Working on a neural pipeline for YOLOv5 myself. That way, feature extraction is optimised for data assimilation tasks
*Thread Reply:* oo will take a look at using this.
*Thread Reply:* @Jason Holmberg (Wild Me) @Sara Beery
*Thread Reply:* EarthVision at CVPR (though the focus in on remote sensing applications at large rather than ecosystems)
*Thread Reply:* This may also be of interest: https://aiforconservation.slack.com/archives/CLWGQ4BJ6/p1702567887929979
}
*Thread Reply:* @Peter Kulits and I have a participatory system that helps collect labeled data with IDs with humans in the loop, if you don't yet have sufficient training data. One of the trickiest things in this space is that there is very little public data (just the ELPephants dataset, which is only 40 individuals and highly curated) due to limitations in data sharing.
*Thread Reply:* A qualitative study in our side seems to suggest that re-ID from camera trap data for many species is only possible for a very small portion of the data even by experts, vs. human captured DSLR data
*Thread Reply:* Hey @Sara Beery and @Peter Kulits! Hmm, yea data accessibility and image quality are tricky. Did you see this paper that the Plotnik research group published in March? https://peerj.com/articles/15130/#aff-2
They're doing everything by human verification of videos from camera traps and getting reasonable seeming results! I was also thinking that, if you could afford it, some of that difficulty in getting high quality imagery from camera traps could be resolved by doubling cameras per location (similar to leopard folks trying to catch both sides of the leopard).
*Thread Reply:* Though they are also subsetting the images they analyze in that paper, and only using the high quality images. Still I feel like you could leave camera traps out long enough to accrue higher quality images for most of the elephants in an area, given enough time. I think the next step, modeling populations, would be trickier though given that variability in data availability.
Also, I don't have camera traps or images yet, but I bet I could find people with that data willing to share for a machine learning project!
*Thread Reply:* Not sure about indian elephant, but on african elephant we use tears and wrinkles in the ears, or the wrinkle pattern from the forehead to the mouth which is unique for each. This allows us to us images of lesser quality
*Thread Reply:* Hi @Paul Allin and @Sara Beery. Thanks for the tips! Could I use that particaptory labelling system that Sara mentioned a little ways down the road? I'm writing a proposal to Wildlabs for money to buy camera traps. Once I have traps, I'm planning to take them to elephant sanctuaries in Thailand. There are hundreds to thousands of captive elephants that free range through forests at least part of the year, with humans that know the elephants very well and can ID them with high confidence. Seems to me like the best possible way to build a training dataset! And I'm writing into the grant that the dataset would be publicly available too!
*Thread Reply:* sounds like a great way to collect data of a large group of elephants. What is the end goal?
*Thread Reply:* the end goal is to be able to estimate populations in and around a protected area in eastern Thailand. I'm hoping individual ID will give better results than gridded camera trap arrays using more traditional analyses
*Thread Reply:* Would be good to chat, I have some ideas for something similar for African elephant and perhaps there is some overlap in ML
*Thread Reply:* Yea, I'd love to chat! Do you have any free time next Monday or Tuesday?
*Thread Reply:* Yes that should work, what time zone are you on?
*Thread Reply:* Which platform do you want to use?
*Thread Reply:* let me shoot you a zoom link
*Thread Reply:* Could we 9am my time, 4pm your time?
*Thread Reply:* Allinpaul@gmail.com, 4pm is fine with me
Hi everyone! It's my pleasure to announce that we finally were able to release a dataset containing images of two species of sea stars: https://lila.science/sea-star-re-id-2023/
The paper that goes with it has been approved but needs some minor tweaks before it can be published (hopefully in the upcoming weeks), I'm happy to share the manuscript with anyone interested in the meantime. And of course, happy to answer any questions about the dataset.
*Thread Reply:* Hi @Oscar W - I'd be interested in giving the paper a read if you're happy to share
*Thread Reply:* Hi Ana, I work with a number of Colombian and Ecuadorian teams. If you are near Medellin, highly recommend check out Juan Parra's lab at Antioquia. Happy to chat further if helpful.
*Thread Reply:* Amazing! Thank you so much, I will contact him
*Thread Reply:* Hi Ana, I am a faculty member at Universidad Nacional de Colombia. My areas of interest include machine learning and data analysis, with a focus on their applications in environmental monitoring. Please feel free to direct message me if you are interested in discussing potential projects
*Thread Reply:* Also just wanted to say that I worked with Ana co-organizing the NeurIPS 2023 Latinx in AI workshop and she was extremely on top of things, eager to push projects forward on her own initiative, had great insights about the different research directions people were working on. Overall the social event we organized had around ~450 people registered which I think says something about her ability to manage large projects :)
For those interested: the Nordic Society Oikos conference in Lund (Sweden), March 12-15 2024, https://nordicsocietyoikos.glueup.com/event/nordic-oikos-2024-80737/, will have a thematic session on computer vision in ecology and evolution https://nordicsocietyoikos.glueup.com/event/nordic-oikos-2024-80737/thematic-sessions.html. The extended deadline for abstract submission is 12th January 2024!
Pleased to share this work with y'all. Enjoy the read! 🙂
https://www.nature.com/articles/s41598-023-49989-z
*Thread Reply:* Hi Dan you could also probably add the EU Rewilding project. Model weights are uploaded on HF
• HF Spaces https://huggingface.co/spaces/skylord/European-ReWilding-Demo-Yolov8
• Model weights https://huggingface.co/skylord/ReWilding-Europe-Yolov8
*Thread Reply:* Added, thanks!
I tested it on an image with deer from my backyard, it did the right thing.
*Thread Reply:* @Dan Morris feel free to update the list of the models for mammals with the paper:
Recognizing European mammals and birds in camera trap images using convolutional neural networks
Two models based on EfficientNetV2 and ConvNextBase + test set are here: https://data.uni-marburg.de/handle/dataumr/246 https://github.com/umr-ds/Marburg-Camera-Traps
Paper: https://inf-cv.uni-jena.de/wordpress/wp-content/uploads/2023/09/Talk-8-Daniel-Schneider.pdf
*Thread Reply:* @Piotr Tynecki Done, thanks! Sample code looks very straightforward. Also added your dataset to this list.
*Thread Reply:* (it’s not my paper/model/code but thanks!)
*Thread Reply:* On the bioacoustics side of model zoos/parks/wildernesses, we’ve been developing the “Bioacaoustics Model Zoo” here. It’s admittedly OpenSoundscape-focused, but it does include APIs for generating predictions or embeddings with Perch and BirdNET. In addition to adding more models to the zoo, the section of the ReadMe “Other automated detection tools for bioacoustics” could be expanded to fit the spirit of “model wilderness” - I’d be happy to work or collaborate on that effort (or port it to another location) if @Dan Morris or others think such a list is needed on the acoustics side of thiings
*Thread Reply:* @Sam Lapp Yes I really like your model zoo! You did not just the easy version, but the proper model zoo that I claimed earlier on this thread is very hard. 🙂 Coincidentally I played with your model zoo for the first time last week, and made my first PR to your repo. Worlds collide!
Hi everyone! Continuing our trend from last year, this time also we have created a 2023 Year in Review Playlist of our group's work (Flight Robotics and Perception Group at Uni Stuttgart). A large part of it is likely of interest for this slack workspace. Enjoy it here : https://www.youtube.com/playlist?list=PLZM-Zi7aahIvGecEUjNafPWhb2pYuwBVO
*Thread Reply:* Wow, in high school and already so motivated... when I was in high school, I think my biggest achievement was beating Final Fantasy (the NES version, in case anyone else here is old enough to make that distinction) in one continuous, pizza-fueled session. And here you are training AI models to analyze the impact of climate change on bird migration. Kudos.
And welcome to this community! If you're looking to "meet" (in the 2023 sense of the word) more folks doing this kind of work, in addition to hanging out on this Slack, I recommend hanging out at as many WILDLABS events as you can; many of us are big fans of the Variety Hour series as a way to get a feel for what's going on in this field that's as close to in-person as one could get without teleporting everyone on this Slack into one room:
*Thread Reply:* Love this recent work compiling these lists Dan, super useful! Do you know of any similar lists for birds (from terrestrial, not aerial, images), insects, plants, or reptiles/amphibians? Or know of any evals of how MD performs for these (I imagine it’s only really performant on birds given the training dataset)?
If these don’t exist, I'm doing some research in this area and happy to compile a first draft to add to the ‘wilderness’.
*Thread Reply:* Sorry, I don't know of similar lists for other terrestrial modalities/taxa.
I don't know of formal evaluations of MD on birds and reptiles, though subjectively I would say for birds, MD works almost all well as it does for mammals, but not quite, and for reptiles and amphibians, it can be anywhere from "excellent" to "a total catastrophe".
*Thread Reply:* "anywhere from "excellent" to "a total catastrophe"." - love it! 😂
*Thread Reply:* And thanks so much! This is really useful
*Thread Reply:* KAUAI AVIAN RESEARCH/MANAGEMENT COORDINATOR – ID# 223806.
CLOSING DATE: January 5, 2024. INQUIRIES: Lisa Crampton
RCUH’s mission is to support and enhance research, development and training in Hawai’i, with a focus on the University of Hawai’i.
We are an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, disability status, genetic information, protected veteran status, or any other characteristic protected by law.
Dr. Lisa “Cali” Crampton
Program Manager
Kauai Forest Bird Recovery Project
PO Box 27 (USPS mail) or 3751 Hanapepe Rd (courier packages)
Hanapepe HI 96716
I know that last year was hard on the planet -- but there was also a lot of innovation and progress in environmental AI and I was inspired by the excellent talks at NeurIPS (especially Sara, David, and Zaira's, just to name a few). I wrote a short recap of some key takeaways in AI4Earth throughout the year. This is not exhaustive by any means, and it is certainly opinionated, and of course any feedback is welcome. https://blog.allenai.org/ai-for-earth-2023-in-review-49a27cb731a8
*Thread Reply:* I wrote this somewhat spontaneously and last moment and wanted it to get out there before the new year -- but I think it would have been much better had it been a more collaborative/jointly written especially to offer more diverse perspectives.
*Thread Reply:* THanks @Patrick Beukema! It is important to keep an optimistic drive and your recap is a great read to start the year with a lot of positive energy!
I get 1-3 requests for review a week at this point. I'm considering a blanket approach that if the journal doesn't require data availability, then I won't review. I encourage others to join me. I'm trying to decide how stringent and reasonable the requirement should be. I think my minimum is that if within 2 min I can find a paper I'd want to read in the latest issue, but the data "Is available on request", I won't review. Use that and the guide for authors as an indicator. As a survey in my inbox I have a request from Ecological Indicators, Biogeosciences, International Journal of Applied Earth Observation and Geoinformation. Ecological Indicators has the typical weak data availability statement (https://www.elsevier.com/researcher/author/tools-and-resources/research-data/data-statement), and I found an interesting paper within 15 https://pdf.sciencedirectassets.com/272637/1-s2.0-S1569843223X00107/1-s2.0-S1569843223004417/main.pdf?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEMr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIQCeq8rVfsuD6kwwap4Ean3c0FY3meO2LFH5h3VNDGTmrwIgeJRmVgOoYa%2FqQNAuNXTRsadLxrrvngtSpvhg01YUaQIqsgUIYhAFGgwwNTkwMDM1NDY4NjUiDEAf6oTFFVTCaYfxCSqPBRC4hAI2d76ydyauTgLwx4Pp%2FYoxeYRP6loc2AGgUNYMGv3kIH%2FPdOIiJtmj1XhOuX446yOAU19QvMj1RRHKr3fCLemamb%2FbbOQ7%2BHmwY%2BR4MctxGMYRr1QV0H%2BMp7EKjgR6e4%2FhWmZ8KlEYFvvK9mnAIPLpIhF%2FTJyKsdX92tST%2BZk7kkKPLY45q9%2FpR3NwvQGv0T73qRvgYKmZOBr3gKC%2BEpddks8WEKJsJER3ROxAcLk9e%2BUljBapaBZS%2FZ3p4YrM7yQCCgZXfaD0ARpoChlDH0KptPOZ2O2WSMWzZK72I%2Bm%2BElYwBluP%2Fai9Lg7ZV2vnxoLX1VEoMHN3%2Fes%2FaSsUTHVVckUsjnNfAGPf7NAZ2bxUa5SgyEpsnbGss6tSpdUhIPini4NZzlewKtg6FizaZR1oi8WBpnrhIhF9OHQGFLcRwe0SPYbyvdiEcwcPcaRsrb4yD5aFs%2BXPY7Q90H1SMAQ3dUM7iJOj%2F4BOVJq7DaYb0m30zbdeiMLezfpnvav87bsItHLU1mD%2BTnY6ZlcFrVA%2F0sd0591Fm%2FHqyLjAVsyESm7gDxzERvTn9b%2F7ZNlnvsMHJR0ggkPJ1e%2FjlM%2FRpVX1v6P70vgYgB9p8uFLKCEMLw28Hbx1dphGyLP0Bbtayjcu6kqpQRWgAfFL%2F8pmpqcQEnMx0771mKRMa%2BYkqnuYN%2BBu%2BbUaDEBJHFR94uFiZhcPGRvxFzsYxdObl8%2Ba2vr8GEiT4mjwoNFAEoTiDcyUbrx1d0wdwbnese%2F95M%2FPD7OM%2BNyXSkHEXmHEf8D%2BuW55L1Qn3xLHp6LASweosb7zK0umAjJrVDVgUb1GPg%2FODODmn0IeBGcphF6cSeaaisIZim6GjJ4EXBPCpK4wsKzWrAY6sQH0jHx7chb%2F7WUXKyvpXrgXT0Wab%2BGhz%2FDUqLC8RczuILfIFeES%2FC8cnPB0Z4EUwAFUirysNIiDgAC2mLcEqy0Ri8YFvOrRni6HlAWAN0EKARkmvwaRldRJY7kBqHrSnBX9lLkNiccvoaIoDfkEyFWghQ43Nq4MlTVy%2FdHNSr9Dnxeh1NHrP7kajCupqYZHVVaqeA3tSIIVlx6hUnYwBpshTwBAJI%2F3blSkY3KECI%2Fs8A8%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240103T175754Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=ASIAQ3PHCVTYVDTMZKDA%2F20240103%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=bf2c855ca6f7e0e03b1d16ee7616c8d58ca58234ac089eda3a59e84691c3e713&hash=e25bf956b2d66971b4143a387f007733879bec598becce1e45a7aec94ec08ed4&host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61&pii=S1569843223004417&tid=spdf-101f8213-f738-4c37-8ada-35d5d0f696d0&sid=1a0126d47157b040387ab0d29247e2cf0c19gxrqa&type=client&tsoh=d3d3LnNjaWVuY2VkaXJlY3QuY29t&ua=17105d520005545e54&rr=83fd20774857ef98&cc=us|seconds. The other journals appear equally vague. Are there other criteria I should use? Is this fair? Does it bias towards more expensive journals and have any other unintended consequences? Of course there are legal and logistical reasons not to share data, but those are the vast minority of cases. A reasonable explanation in the data availability section is usually sufficient in those cases. FYI, credit to Methods in Ecology and Evolution, which has probably the most comprehensive statement (https://besjournals.onlinelibrary.wiley.com/hub/editorial-policies)
Data are important products of the scientific enterprise, and they should be preserved and usable for decades in the future. The British Ecological Society thus requires, as a condition for publication, that all data supporting the results in papers published in its journals are archived in an appropriate public archive offering open access and guaranteed preservation. For theoretical papers the underlying model code must be archived.
*Thread Reply:* I think this is reasonable -- (lack of) dataset availability came up several times for me while reviewing for NeurIPS Datasets and Benchmarks and AAAI AI for Social Impact tracks. It is impossible to properly review some papers without data availability.
*Thread Reply:* I generally agree with what Ben says (everyone is almost certainly tired of hearing me shout about public data on this Slack), but I have a slightly softer approach as a reviewer...
- If there's a good reason that data can't be released, and it's either mentioned specifically or it's self-evident, I don't totally ignore the issue as a reviewer, but I look at it as slightly narrowing the audience that this paper might impact and slightly raising the bar for reproducibility. There are a zillion other things that make an audience narrow or broad, or make it more/less difficult to reproduce a result, and this goes in the pile somewhere, not even really as a strike against the paper, more as a bonus point they didn't get from me. The most common reason I see is licensing of commercial satellite imagery... but that's also an example of where the models/methods are still useful to other people, because if a paper says "we used WorldView-3 imagery but we can't release it", readers can still purchase WV-3 imagery and make use of the methods/models from the paper. I.e., this is really different than saying "we built a totally custom hyperspectral drone you'll never see and here are our results that you can never verify". I think we don't want to create a culture where no one can ever write papers about AI methods for commercial satellite imagery. Ditto for papers that use data that came from sources that don't allow public release; in the camera trap space, for example, requiring data release would in practice bias significantly against writing papers about data from indigenous land, or from whole countries (e.g. India) where data release is hard or impossible, or at least much harder than it is in the US.
- If a paper doesn't release the data and doesn't really say anything about why, that's a more significant strike from me, because I agree with your inclination toward encouraging data release, but it's still not a dealbreaker, just an imperfection. If a paper is otherwise awesome, with awesome open-source code and/or public models and/or methods that are obviously a good idea, I might still give an above-the-rejection-line rating.
- If the paper says that part of their contribution is that the data is publicly available, and I click the link and it's obviously not really available, now I get really cranky as a reviewer. As often as not it's just carelessness, and maybe they intend to really release the data, but IMO you can't have it both ways. Same for the case where data is "publicly available" but I have to log in to access it during the anonymous review process. Grrrr.
*Thread Reply:* I totally agree with these guidelines for once you accept the review. I'm just talking about trying to choose among the many review requests we are all getting. These are all good points. Usually you just get an abstract, so you can't even tell any of these things yet.
Hey guys, really bad at checking this channel but FYI our Indigenous-led biodiversity credit methodology is in open review now and will be the first certified biodiversity credit in the world ~ Feb. If you’re a biodiversity nerd like me please consider giving us a
Sadly we’re not eligible for this one (marine monitoring is out of scope), but here’s a new funding call from InnovateUK:
> Defra and Innovate UK will invest up to £5 million in collaborative innovation projects that develop environmental monitoring solutions. > Projects must develop new or repurpose existing sensor systems and capabilities, for example observation systems, sensor or sampler carrying platforms or modelling systems and focus on one or more of the following challenge areas: > - biodiversity and natural capital > - soil health (including measuring soil carbon) > - water quality > - greenhouse gas (GHG) and ammonia emissions from Defra sectors
*Thread Reply:* As promised, let me convince you that camera trap datasets don't exist on LILA.
That is, camera trap "datasets" still exist on LILA, but I encourage folks to think of them as arbitrary divisions within the one big dataset that is "LILA camera trap stuff". E.g., if you want images of coyotes, you probably don't care whether they came from the "Idaho Camera Traps" dataset or the "Caltech Camera Traps" dataset.
So for most ML training applications, rather than dealing with individual data sets, consider working with the Really Big CSV File that contains all the camera trap images on LILA, mapped into a common taxonomy:
https://lila.science/taxonomy-mapping-for-camera-trap-data-sets/
The Really Big CSV File has also been imported as a Hugging Face dataset:
https://huggingface.co/datasets/society-ethics/lila_camera_traps
*Thread Reply:* Also, if you're wondering "how many images did Dan look at to find the penguin doing the penguin-i-est thing a penguin could be doing for that thumbnail?", the answer is... maybe 5,000? Worth it.
*Thread Reply:* I also see that @Joris Tinnemans, who did all the hard work to collect and organize this dataset, has joined this Slack... welcome Joris, and thank you for your contribution to LILA!
*Thread Reply:* cc: @Peter Bull (spoke to him recently about this space and he wanted to learn more given his recent relevant work)
*Thread Reply:* Not relevant to your question, but I didn't realize until you mentioned it on this thread that NACCB is in Vancouver this year. I never go anywhere, and I consider Vancouver not going anywhere (from Seattle), so I'm in! Excited to meet folks at NACCB. Thanks @charlotte for pointing that out!
*Thread Reply:* Thanks @Dan Morris and glad to hear! NACCB is a great venue for meeting folks in the policy space as well — the previous NACCB meeting featured a large “America the Beautiful” (aka 30x30) US eNGO meeting where scientists and policy leaders in these orgs were triangulating their strategies for landscape prioritization, communicating to policymakers, etc.
*Thread Reply:* I'll be there as a speaker in another symposium, looking forward to see your symposium!!
*Thread Reply:* Happy to help. We have been working in this area for a few years. We have an RGB tree algorithm (https://deepforest.readthedocs.io/en/latest/), that has been fine-tuned for tropical forests https://ieeexplore.ieee.org/abstract/document/9387530 (but needs more annotations). 4 or 5 papers on tree species classification in hyperspectral data (https://scholar.google.com/citations?user=7POnELAAAAAJ&hl=en) and we are working on a global tree detection benchmark https://milliontrees.idtrees.org/. We would love to help and have your data included, every dataset gets the community closer to a usable baseline. RGB only classification is hard, but I can point to several interesting papers (a couple attached).
*Thread Reply:* Wow, thank you Ben for that rich vein of information. I look forward to reading those references and I’ll respond with questions when I have done so.
*Thread Reply:* Would suggest reaching out to the Forest Restoration Unit in Chiang Mai, Thailand as they have recently published on this as well http://forru.org
*Thread Reply:* https://lila.science/otherdatasets#bioacoustics
*Thread Reply:* https://bioacousticsdatasets.weebly.com/index.html
*Thread Reply:* do you mean like labeled datasets? raw soundscape recordings? completely open-source you-can-download-all-files? all of the above?
*Thread Reply:* Good questions! I guess I mean anything that has already been somewhat curated/selected, and open data. My secret motivation is... I have new PhD students starting this year, and I'd like it to be clear for people like them to know where to find "all the good new bioacoustics datasets we could use in our ML"
*Thread Reply:* I'd be very interested in a curated list of audio datasets of animal sounds as well!
Where would I look? I would personally do a google site:github.com search and look for a GitHub repo which maintains a curated list of animal sounds (did not find any). However, there seems to be a curated list of audio technologies (https://github.com/DolbyIO/awesome-audio, including some audio datasets at the end). When this fails me I would probably just do a google search and see what I find. I may also look at: https://dcase-repo.github.io/dcase_datalist/ which would lead me to the list by Justin that Carly linked (https://bioacousticsdatasets.weebly.com/). I'd completely miss the lila.science list with this approach. Thank you Carly for that link!
Personal thought: There are a lot of lists on GitHub with the "awesome-
Maybe this is an opportunity to create a "awesome-animal-sound-datasets" repository?
*Thread Reply:* There is also another list that would be of interest, which is that of open-source models. For example, the Kitzes lab (@Justin Kitzes @Tessa Rhinehart @Sam Lapp) is working on a bioacoustics model zoo - https://github.com/kitzeslab/bioacoustics-model-zoo/tree/main
*Thread Reply:* https://wildtrax.ca/ has almost ~750K acoustic recordings where the processed data (the first detection of all species identified by human expert human listeners) is public. You can download flac clips of those detections for model training
*Thread Reply:* there are 10 collected here: https://github.com/earthspecies/beans
*Thread Reply:* Since it hasn’t been mentioned: https://xeno-canto.org is an awesome call repository for birds, and has some insects and bats as well
*Thread Reply:* here’s one list more by Tessa, with contents from Justin’s and Dan’s https://docs.google.com/spreadsheets/d/1KrmCB0vvSK7V3znJfycO-eOMZJKP2F-Ih6neRYPz1Xc/edit#gid=0
*Thread Reply:* Another good one is https://www.inaturalist.org/. I used it to get recordings of squirrels and amphibians for my bird sound recognizer: https://github.com/jhuus/HawkEars. It has a nice API too: https://github.com/pyinat/pyinaturalist
*Thread Reply:* @Ben Koger is great! apply with him!
Question for the group especially anyone who might know folks at ESA. We are working on a new dataset that depends on WorldCover ESA — WorldCover ESA did not include Antartica, but i came across this press release from Copernicus about a cloud free mosaic (3000 Sentinel-2 images) covering Antarctica: https://sentinels.copernicus.eu/web/sentinel/-/copernicus-sentinel-2-pieces-together-mosaic-of-antarctica That page says “the data are free and open to all scientists and researchers” but I can’t find any link or any other reference to that dataset. Does anyone know about that data or works at ESA who might be able to help?
*Thread Reply:* It might be that one is simply supposed to use the Copernicus browser — but the press release made it sound like there would be some specific data product https://dataspace.copernicus.eu/browser/?zoom=7&lat=-77.75217&lng=-175.61199&themeId=DEFAULT-THEME&visualizationUrl=https%3A%2F%2Fsh.dataspace.copernicus.eu%2Fogc%2Fwms%2Fa91f72b5-f393-4320-bc0f-990129bd9e63&datasetId=S2L2ACDAS&fromTime=2023-01-01T00%3A00%3A00.000Z&toTime=2023-02-28T23%3A59%3A59.999Z&layerId=1TRUECOLOR&mosaickingOrder=mostRecent&demSource3D=%22MAPZEN%22&cloudCoverage=9&dateMode=TIME%20RANGE|https://dataspace.copernicus.eu/browser/?zoom=7&lat=-77.75217&lng=-175.61199&themeId=DEF[…]urce3D=%22MAPZEN%22&cloudCoverage=9&dateMode=TIME%20RANGE
*Thread Reply:* i know this wasn’t wildly popular but in case anyone else ends up looking for it, support at Copernicus kindly led me to this: https://s2gm.land.copernicus.eu/mosaic-hub
Hello everyone! I wanted to share with everyone the video of our expedition in Djibouti :flag_dj:, where with a team of data scientists and marine ecologists from Switzerland, Sudan and Djibouti we went studying the status of coral reefs 🪸 at the entrance of the Red Sea. It was an inspirational experience and we are now analysing the data, more news will follow! For now enjoy the beauty of Djibouti! PS: if you are on a schedule, we talk about AI starting minute 6:40 with @Jonathan Sauder 😉
https://vimeo.com/903195229?share=copy
The 4th CV4Animals workshop will take place at CVPR2024 in Seattle!
We invite submissions in 2 tracks:
- short 4-page unpublished work (potential invitation to IJCV Special Issue)
- published work Deadline: March 27, 2024
*Thread Reply:* Hi Ben, I would rather use some old-school Computer Vision methods than train some large deep models. You can look for Template Matching, Thresholding, Moments, and Moment invariants. There are already multiple methods implemented within the OpenCV. I hope this helps.
*Thread Reply:* Definitely try segment anything https://segment-anything.com/demo
*Thread Reply:* Good idea, @Ben Weinstein. I just tried it, and the objects are apparently too small for the SaM. I might have done something wrong, but it did not work as intended.
*Thread Reply:* Does it work on a zoom crop?
*Thread Reply:* Yes should've added I tried SaM too! It does poorly on the full pic, but splitting into smaller pics it does ok. I'm a SaM noob so need to look into off the shelf stuff that can build on this (e.g allow labelling of segments and train a model).
@Kieran McCloskey
*Thread Reply:* https://labelstud.io/integrations/machine-learning/segment-anything-model/
*Thread Reply:* Maybe something more old-school with erosion+dilation+edge detection+thresholding+contouring could work here? https://www.youtube.com/watch?v=DsePM4F3tEw
https://shrishailsgajbhar.github.io/post/OpenCV-Apple-detection-counting
*Thread Reply:* Hi @Ben Williams, perhaps SegmentEveryGrain could work here? It's an offshoot of SAM that claims to focus grain-like objects. If I remember right a colleague of mine tried using it for ice floe segmentation, though I can't remember how well it worked
*Thread Reply:* These are super useful, thank you! SegmentEveryGrain sounds ideal, will pass it all on
*Thread Reply:* def report back on SegmentEveryGrain, we've been thinking about retraining SAM for tree crowns and I don't know how worthwhile fine-tuning is, versus just using vanilla.
*Thread Reply:* @Ben Williams I am curious if you were able to count the number of grains. I got a grand total count of 39,690 grains.
The workflow is provided in this github repo: https://github.com/Think-Evolve-Consulting/Counting-grains and a short blog: https://www.thinkevolveconsulting.com/2050-2-counting-grains/
*Thread Reply:* Thanks Aakash this is awesome! I won't be trying it but my colleague will so I'll pass it on., this could definitely be useful. Perhaps we're onto a new grain counting benchmark!
*Thread Reply:* Also new to this but very interested in and trying to get involved! Got any key papers or particular tech you're excited about?
*Thread Reply:* just reading this meta paper to start with !
*Thread Reply:* https://www.mdpi.com/2504-446X/3/1/9
*Thread Reply:* I work a bunch in this area. Happy to connect you with labs, are you a developer, have a target use case? Kind of organism you are interested in?
*Thread Reply:* fab ! - yes im a developer - currently interested in habitat assessment using drones / GIS .
*Thread Reply:* hey @Jonah Fox & @Ben Weinstein - im also a developer and have actually recently launched a business to work on AI for biodiversity with a small team of AI devs would love to stay in touch & chat about any specific projects / collabs! atm we're working on wildlife camera trap stuff and GIS for analysing forests to predict carbon abatement
*Thread Reply:* I am interested.
Especially if there is a way to identify metallic traps that are placed by poachers in dense forests. Sometimes these are electrified, leading to significant stress to the trapped animals.
*Thread Reply:* Hey all, we're just getting up and running but you may be interested in our work at Open Forest Observatory. We're developing easy-to-use workflows for processing drone imagery for forest ecology. My main project is the multiview mapping toolkit which allows you to do deep learning model training and prediction directly on individual drone images instead of stitched orthomosaics.
*Thread Reply:* @David Russell @Casey Clifton - would love to have a chat some time ! Are you around this week ?
*Thread Reply:* @Jonah Fox that would be great. Would Thursday or Friday morning (say 9-12) EST work for you?
*Thread Reply:* let's go for Thursday at 9 EST if thats ok
*Thread Reply:* I'll have to pass this time but will check out the above links!
*Thread Reply:* In 2023 and before it was possible to get Slack for free workspace but with Paid plans features (like unlimited history). There are two options for get that: Slack for nonprofits and Slack for education but I see they offer 85% of discount this year.
*Thread Reply:* Same! Saving for later used to be a work-around.
*Thread Reply:* until last year, slack would keep a certain number of messages (megabytes) in the free version and delete the oldest when the space limit was reached. it would not delete pinned or saved messages. They changed that. Everything older than 90 days is now hidden, including saved and pinned messages. it sometimes shows it blurred out to tease people to buy the paid plan, internally the history is kept, but inaccessible unless you pay.
*Thread Reply:* i wonder if the 90-day-limit applies to the self-DM? i sometimes put things in there.
*Thread Reply:* self DMs are apparently not affected when they are text, but the limit does apply to files and attachements - those get a placeholder that they are hidden because they are older than 90 days
*Thread Reply:* I've looked into this several times, but don't have any easy/good solutions. If someone has the capacity to come up with a plan for how to switch this to something that maintains history and how to pay for/maintain funding support for it I would be very welcome to someone leading that charge! Issues I've run into in the past are that with large communities like this the cost is still high even for nonprofits/education. And there are some limitations to both that end up being restrictive (ie we're not all at one institution)
*Thread Reply:* @Sara Beery should we consider reaching out to organizations like WILDLABS for support? They could potentially advocate for the community and help us secure Slack for nonprofits, with us covering the remaining 15% of the monthly budget.
*Thread Reply:* They already have their own internal slack that is using their nonprofit ID, and I think you only get one per nonprofit. We could also start a nonprofit, but that seems complicated as well 🙂
*Thread Reply:* another option is to go back to non-proprietary chat options (XMPP based,
*Thread Reply:* i am a fan (and user) of matrix/element - basically use it as a slack/discord substitute for a couple different groups i am part of. but it seems to be a big ask for folks to adopt and learn yet another app. 😕 i wonder if we could find an ai/conservation ngo which does not already use slack (and never wants to!) who might want to sponsor us. 🤔
*Thread Reply:* Discord and MS teams are free and maintain history indefinitely, but those require shifting platforms
*Thread Reply:* @Chris Yeh these don't solve the problem though - of a single 3rd party provider having control over both server/protocol, client/app and the terms of service. We'd be back to the situation with slack before they changed the rules, hoping they wouldn't change the rules (charge more money/limit features) or like google chat - stopping the service altogether.
*Thread Reply:* Let's go back to IRC. Slack, Discord, Teams, etc. are all just modern attempts at reinventing it anyway. 😄
*Thread Reply:* yeah, IRC on freenode 🔥, I spent more than 10 yrs with the black screen
*Thread Reply:* Every non-profit / academic Slack I know has been struggling with this. Some are limping along with the truncated history, while others have switched to Discord. Really sad Slack changed their free model, and it is completely unrealistic to get the paid plan for a large but informal academic group like this.
*Thread Reply:* Discord has its advantages. Because its primary audience is the gaming community, they're likely to lose a large part of their audience by making it paid to use.
*Thread Reply:* yeah but the gaming community would let them get away with other things the professional field wouldn't - I could totally see them do stuff like adding advertising revenue based on what topic is being discussed in chat
*Thread Reply:* unless it causes lag in games, that'd be their death sentence
*Thread Reply:* I've been part of a few (pseudo-) academic Slacks that shifted away to Discord for this reason - if there's enough buy-in from the community on here, shifting platforms might work for the better.
*Thread Reply:* Hi! We at Rainforest Connection and Arbimon have lots of projects in SE Asia using sound and AI for biodiversity monitoring! Feel free to email me if you’d like to talk more 🙂 carly@rfcx.org
*Thread Reply:* Sawadee krap Lyn! I'm doing my PhD work in Thailand on elephant impacts on forests. I'm going to be doing some drone work and camera trapping, hopefully starting this summer. I'd love to connect and collaborate on some outreach material down the road! Shoot me an email at brandon.hays@duke.edu
*Thread Reply:* Thank you for your replies @Carly Batist and @Brandon Hays! I will drop you an email now. Excited to hear more 🙂
*Thread Reply:* Hi, happy to have a chat. I am involved in https://www.conservationai.co.uk/ Best, Serge
https://hiimlinn.wixsite.com/lynnanticha
*Thread Reply:* Not sure if this is what you mean by metadata, but the Global Biodiversity Information Facility has both occurrence data and species fact sheets for a lot lot of species https://www.gbif.org/
*Thread Reply:* I help maintain a biodiversity dataset repo. You can check it out for plant databases, and feel free to ping me for additions https://earthskysea.org/biodiversity-databases/
*Thread Reply:* Perhaps https://www.worldfloraonline.org/ or https://powo.science.kew.org/
*Thread Reply:* You're always welcome to come visit Carly!
*Thread Reply:* Do you mean photos from cell phones of individual trees? Or photos of forests from above. What’s the use case, we have collected a lot.
*Thread Reply:* I mean terrestrial photos like from cell phones
*Thread Reply:* Hey @Autumn Nguyen, If you're interested in working with camera trap data (images/videos) for an NGO based in Europe and implementing state-of-the-art AI/CV algorithms to address specific issues, please feel free to reach out to me. I'd be happy to collaborate.
*Thread Reply:* This time I’ll join, sara (assuming that i am traveling), promised
*Thread Reply:* It's always fun to watch the nightly crow migration at UW. Around 10k crows fly in every night to a communal roost. https://environment.uw.edu/news/2021/12/a-story-of-10000-crows-the-nightly-migration-to-uw-bothell-campus/
*Thread Reply:* Fomo fr
*Thread Reply:* I'm also a fan of seeing the crows descend on Bothell, but only during the winter, because I'm old and go to bed early and it turns out that crows respond to sunlight and don't use clocks.
*Thread Reply:* If anyone wants to come over to Redmond (~15 minutes without traffic) really early in the morning, you can visit me in my native habitat, which is the Marymoor Dog Park, which doubles as a suburban wildlife sanctuary. I can't guarantee herons, eagles, ospreys, kingfishers, or beavers, but if you come here around sunrise, I can give you as close to a guarantee as you'll get in wildlife viewing. I can 100% guarantee a golden retriever sighting.
*Thread Reply:* Not birds, but since the convention center is a short walk from the waterfront, it might be possible to spot some orcas from downtown: https://www.oregonlive.com/pacific-northwest-news/2023/10/want-to-see-an-orca-from-the-seattle-shoreline-theres-a-group-chat-for-that.html
*Thread Reply:* Union Bay Natural Area by UW campus is excellent and is one of the most heavily birded areas in the region. I'd love to join!
*Thread Reply:* NYT's "Opinion Today" title is "Birding can change your life" -- https://www.nytimes.com/2024/03/30/opinion/birding-spring-merlin-ebird.html. Made me think about this thread :)
*Thread Reply:* Hey all, The deadline for applications to the CV4E 2025 winter workshop has been extended till Monday April 1st! Please consider applying, everyone knows the best thing to do in winter is to sit somewhere sunny and write awesome AI for Conservation code 🙂 See more info in the thread above.
*Thread Reply:* Hi ! You mean for the task of Re-ID or species classification ?
*Thread Reply:* species classification
*Thread Reply:* @Aamir Ahmad feel free to take a look on the Publicly-available ML models for camera traps website powered by @Dan Morris.
You can consider to try DeepFaune or Marbug Camera Traps models for testing, if sets of species are valid for your case.
*Thread Reply:* thanks!! actually, i needed this summary info on approx how many and which species are automatically classifiable. but i will look at those links. Thanks.
*Thread Reply:* I don't recommend quantifying the number of species that can be classified this way; the number of classifiable species is extremely specific to different modalities, different taxa, different ecosystems, etc. And maybe more importantly "can be recognized" squashes subtle questions of accuracy into "can/can't".
That said, I can't help but try to answer the question of what model wins the gold medal for "most species recognized in images", and I'm almost positive it would be the iNaturalist computer vision model.
According to the most recent blog post I can find about their model:
https://www.inaturalist.org/blog/83370-a-new-computer-vision-model-v2-6-including-1-399-new-taxa
...it has ~78k total taxa, and it looks like somewhere on the order of 50% of those are animals. Let's round off to a neat 40k. So, I think the gold medal goes to the iNat model with ~40k animals?
There's an interesting question about how many species "can" be recognized by a multimodal model like Gemini that isn't specific to biodiversity applications. Maybe Gemini has seen more than 40k animal species in training, I don't know. But I think the spirit of the question is "among models whose raison d'etre is recognizing animals...", so, I'm going with iNat.
*Thread Reply:* fwiw, our production platforms at wild me each cover multiple species. thus, ~17 server/sites, but (likely more than) 54 species. note, however, that not all these species are able to be classified via ML. (e.g. our algorithms do not make a distinction between several giraffe species). so this number is very likely not a good measure of species classification you are looking for.
*Thread Reply:* BioCLIP might be of interest: https://imageomics.github.io/bioclip/
*Thread Reply:* hi @Rowan Converse! thanks for sharing this dataset! are you able to share the drone telemetry information associated with the data?
*Thread Reply:* Hi, none of the webpages loaded... I was using chrome? I'm interested in this, as have hundreds of sticky traps with insects from the southwest with no feasible plan to id them.
*Thread Reply:* The links are also not working on safari and chrome on mac…
*Thread Reply:* Yeah... It seams like a cyber security action our IT took, they are informed and probably restore abroad communication by the end of the weekend. Here you may find a youtube playlist with demos. A Colab notebook of the AI training. A demo data set from our database. And the Gitlab group.
Those links can be found on our website too, once it will be available to others outside the university.
It is still under development, and for the safety of the data we and our colleagues gather, to use the tool you need to approach the admin to get username and password,
Have a great weekend!
*Thread Reply:* The admin: Prof Chen Keasar - keasar@bgu.ac.il
*Thread Reply:* Just to inform all, the cyber attack we were under for a week or so, is behind us. The site is back and active,
*Thread Reply:* thats great news! ill check it out
*Thread Reply:* Awesome, thanks for the update. I look forward to trying it out!
*Thread Reply:* Hi Cynthia, I saw this message now. Were you able to resolve this already? I am also curious about this and would love to know your findings
*Thread Reply:* Thank you so much Evan for such a thoughtful and helpful response! We’re going to try some of these and see how it goes. Really appreciate it!!
Our friends at Silverstrand are launching applications for their Biodiversity Accelerator... if you're building a biodiversity focused venture at pre-seed/seed stage, this is for you! Can vouch for how awesome these folk are... https://www.linkedin.com/posts/silverstrand-capitalimpact-innovation-capacitybuilding-activity-7165515765654171648-ZArZ?utmsource=share&utmmedium=memberdesktop|https://www.linkedin.com/posts/silverstrand-capitalimpact-innovation-capacitybuildin[…]765654171648-ZArZ?utmsource=share&utmmedium=memberdesktop
Two week environmental hackathon coming up in May with ETH AI Center: https://hack.biodivx.org/
*Thread Reply:* Thanks for sharing this Dr. Beery! Seems like a great opportunity, and I would love to pass it on to others. Do we know if this is exclusively an in-person event? I can't seem to find a distinction between in-person and virtual participation.
*Thread Reply:* I'm not sure! You could reach out to the organizers and ask maybe?
*Thread Reply:* Thanks for sharing, we are aiming to make it hybrid!!! @Ronan Wallace 🌿
*Thread Reply:* Welcome, Davide, and thank you for participating in the AI for Bears challenge!
*Thread Reply:* Thanks Ed! I'm very happy to be able to contribute to this project 🙂
*Thread Reply:* wow just checked them out - really nice projects!
*Thread Reply:* @Lasha Otarashvili @Jason Holmberg (Wild Me)
*Thread Reply:* @David Johnston moonshine could be cool for coral reef segmentation
*Thread Reply:* Re: Moonshine...
While everyone else is complimenting the Moonshine library, I'd like to take a moment to compliment your logo. I see you, "O" shaped like a moon.
Re: drones and conservation...
You may find this page useful; I try to keep track of open datasets and open models in this area:
https://github.com/agentmorris/agentmorrispublic/blob/main/drone-datasets.md
I'm not saying you have to go train a big model on all that data and put it in the Moonshine package, but if you were already thinking of adding object detection... just saying...
Re: music...
If you're ever in the PNW, we can jam and talk about conservation technology; my real contribution to society is:
*Thread Reply:* Hi Nate, welcome. We have worked in this area for a couple years on tree detection, bird species classification from airborne imagery. A quick list of challenges for the field might include • Generalization across image resolution for multiple data acquisition systems • Cross-sensor calibration for remote-sensing with different numbers of bands • interactive tools for batch labeling across hundreds of images using unsupervised clustering of embedded features • cross-geometry prediction of different annotation types, so going between point/polygon/boxes to make sure we have access to every data source we can get (see SAM-geo). We work on a number of benchmark datasets for tree detection and developing more (https://milliontrees.idtrees.org/), and release our models through a python package https://deepforest.readthedocs.io/en/latest/prebuilt.html. There are large datasets that speak all these challenges, if interested, or anyone else here. @Devis Tuia’s team works a bit more on some of the robotics/engineering challenges through their new collaborative that is really interesting https://wilddrone.eu/
*Thread Reply:* Wow thanks for the warm welcome everyone, it’s super cool seeing what everyone is up to! I might reach out to some of you to try and learn more about what you’re doing, but also feel free to message me if you wanna chat sometime!
*Thread Reply:* Hi Nate! Cool projects.
I work with the Wild Me team for aerial surveying with the Scout (UI) & Scoutbot (ML plugin) products. Currently Scout is being used for aerial surveys in Africa in Kenya and Kavango-Zambezi transfrontier, with expansion work underway. Two challenges we are currently working to solve: • Imbalanced data - given the long-tailed distribution of species, trying to predict the objectness along with the species label often reduces performance for objectness, even with a decoupled head. • Dense images - tight herds. Two problems - obtaining ground-truth labels is hard when there are 1000+ objects present in a single image. Models trained on less dense images have a hard time stepping up. I feel solutions will be outside of localization-as-bbox approach. Happy to chat if interested.
*Thread Reply:* @Dan Morris, awesome list! I'm bookmarking some of these to look at later. Also, KABR is actually available on Hugging Face (along with the telemetry data from the flights).
*Thread Reply:* Glad the list is useful. And thanks for the pointer, I added the HF link to the listing for KABR.
*Thread Reply:* Welcome @Nate Harada! I have been working on tree monitoring at Taro AI 🌳 (taroai.com). So cool what you’re building at Moonshine, it looks so useful! I’m also located in SF if you ever want to go for a walk!
https://twitter.com/WildAudioJack/status/1764595424608821691
*Thread Reply:* can you show us a few images? Draw the thing in the image you want as a output.
*Thread Reply:* Hi, attaching some images. What do you mean by what I would want as output?
*Thread Reply:* machine learning models take in data and return predictions do you want to 1) label each entire image with a species -> classification, 2) Label each individual in an image to species -> detection. If you want to classify them, how many images do you have of each species? Can you go to other data sources and get more from citizen science (iNat?). @Justin Kay what's the latest on the fish foundation models. @Mai Lazarus do you know any R or python, or are you look for a desktop application?
*Thread Reply:* Hi, I would like to get a classification, the identity within each picture is not as important (I would like to crop images with several individuals to have as many images as possible, if the resolution is good enough). I have about 100 images and for sure I can get much more from other data sources. I am quite experienced with R. Thanks for the help!
*Thread Reply:* probably start here. https://cran.r-project.org/web/packages/keras/vignettes/ You'll probably need atleast 50 images for most classes to get a decent model. I don't know the fish space as well, hopefully someone can jump in and say if we have a backbone model to start training from.
*Thread Reply:* @Lasha Otarashvili
*Thread Reply:* My go-to recommendation has been "ask Ben Weinstein for his Label Studio template and do exactly what he did"; I'm not sure what I should make of the fact that my recommendation for Ben's setup is stronger than Ben's recommendation for Ben's setup. But FWIW, Ben, what you're doing felt like exactly the right amount of not reinventing the wheel, but also exactly the right amount of not taking dependencies on complicated ML features in Label Studio (rather just having good tools for fine-tuning and running models outside of LS, e.g. overnight, and loading the results into LS). So IMO a great starting point would be for you to document what you've done with Label Studio to the point where others could try it. Eh?
*Thread Reply:* Nice list above! We have been actively developing SmarterLabelme https://github.com/robot-perception-group/smarter-labelme, for many of the listed tasks, focusing on aerial videos with many individuals in the view. We are using it for detection, tracking, label propogation, etc. but also a lot for quickly auto-annotating behaviors... feel free to check out or ask me about it. The behavior version is here: https://github.com/robot-perception-group/animal-behaviour-inference
*Thread Reply:* That's great, can you talk alittle about what the largest time investments were? What have you learned so far?
*Thread Reply:* Sure. It depends on the exact task, but typically one would spend most time in the initial expert labeling to get a good detection model. If such a model exists (which is the case for several species now), then you gain speed even there. For behaviors classification, for example, we spent a couple of days to annotate some initial sequences. Then used it to learn a classifier, which was then used to auto annotate (using the framework) to 'quickly' annotate more and more and then use those many new annotations further improve the accuracy of the model. What we have learned -- one thing, at least for the behavior classification task, that stands out to us is that some behaviors are very 'temporal' in nature and hard to learn from 'still annotations'. Another thing is that to use the auto-generated annotations for retraining, one has to still go over the annotations to correct the few wrong ones. Even though this is super fast than what it would take to annotate all those videos in the first place, it is still unavoidable.
*Thread Reply:* Great! but I meant what the time investments were for designing the system. What took the most time to create, using QT, image backend, model deployment?
*Thread Reply:* @Eric Price can answer that best
*Thread Reply:* Hi. We, too extended an existing open source tool for annotation and integrated models for detection and tracking into it, so we spent almost no time on the user interface, except adding a few buttons/key shortcuts for the new functionality. Maybe 2 days to understand the code and API, and one more day extending GUI. - but depending on the complexity and functionality of the tool that can be more elaborate, especially for web based tools. The main work was definitely model deployment - on the order of weeks, but thats mostly because we combined multiple models and extended them beyond what the base models could do on their own so there was new functionality. Deploying an existing model should be much quicker 1-2 days maybe. it depends also if the frameworks are compatible. in our case the GUI was python and the model was written in pytorch, so we could just call it directly, but there's combinations where that could be more time consuming, especially for cloud based tools.
*Thread Reply:* does that answer your question?
*Thread Reply:* @Ben Weinstein been meaning to email you back and set up a time to chat. In two weeks?
*Thread Reply:* for sure, I know you are super busy, take your time.
*Thread Reply:* I feel like I should also mention VIAME here @Daniel Davila https://www.viametoolkit.org/
*Thread Reply:* wow some amazing stuff here guys!
*Thread Reply:* I'm interested in tagging of aerial data for habitats. E.g. "road", "tree", "hedgerow" and then more specific if there's the detail - e.g. specific species - or "species rich hedgerow" Does anyone know of any work in this area in particular ?
*Thread Reply:* Something that could be useful - dealing with image overlap. Many fly-by styles that do continuous collection produce images that are partially overlapping - making it possible to overcount the bounding-box predictions unless dealt with. Scout has a feature to draw a straight line to discard an area in the image that appears elsewhere. This works well for plane-mounted cameras as they fly in long straight lines - so there is only one axis of overlap. Drone-mounted cameras might produce overlap in both dimensions - think of a reference image having 4 adjacent overlapping images from N,E,S,W.
*Thread Reply:* Great to see you here Sachin!
*Thread Reply:* Hi! it seems like your classifier has a lot overlap in species with my camera trap project canyoncritters.org I'd be interested in applying it if its available
*Thread Reply:* @Anna Willoughby Hi, I would be happy to help. I am not much active on slack, but you can send me an email (sachin27071998@gmail.com) and I can share the model weights that can be used with animl-py repository. https://github.com/conservationtechlab/animl-py
*Thread Reply:* When I was at Google, I got started by doing a 20% project in climate!
Hi All, I just heard about this group and joined. Our research team at the University of Oregon is developing self-supervised models (BERT related) to automatically parse animal vocalizations. I described some of our preliminary work in this recent talk for the Interspecies Internet - link below. I hope that we can apply these models to conservation challenges in addition to research on animal vocalizations. If we consider models built to detect the presence of a species in environmental recordings, I think it will be interesting to research what kinds of self-supervised pre-training processes lead to the most sample-efficient fine-tuned models for conservation... Happy to chat! https://www.interspecies.io/lectures/timgardner
*Thread Reply:* Hi Tim, so interesting to hear about your research! There are quite a few of us at Oregon State University who are using bioacoustics to detect birds. It would be great to connect sometime.
*Thread Reply:* I'd love to connect! Feel free to reach out to timg@uoregon.edu and we can schedule a time to talk.
*Thread Reply:* Not sure if that's what you are looking for: We developed an open source tool for video annotation (smarter-labelme). It can do classification, object detection, but only limited segmentation. For bounding box annotation, it uses inbuilt detector and trackers to streamline the annotation process (the tool makes annotations and the user corrects them as needed). Its using python+QT and is not web based, ideally using a computer with GPU for running the detection/tracking models. There's no inbuilt data or user management though, beyond keeping track of the annotated frames per video in the form of json files, We handled that by handing out individual video files to annotators and then collecting the annotation data on a shared folder. https://github.com/robot-perception-group/smarter-labelme
*Thread Reply:* Thanks @Eric Price... @Petar Gyurov there is a recent discussion on this in a thread 2 messages above in the general channel!
*Thread Reply:* linking to above thread https://aiforconservation.slack.com/archives/CLWGQ4BJ6/p1710264474399949
*Thread Reply:* Thanks, I wasn’t able to see the previous messages in the channel for some reason but I can now. @Eric Price , Smarter-Labelme looks great, but I am not sure it fills the gaps that my team faces in LabelStudio. Will definitely keep an eye on its development though.
*Thread Reply:* Thanks. If you know any features that you'd like to see implemented or think are missing, feel free to tell us 🙂
*Thread Reply:* @Petar Gyurov Which of those boxes does the self-hosted Label Studio not check? I'm not claiming it's perfect, it just seems to check those boxes, and I've been impressed with it to the degree that I've tried it. My only gripes with Label Studio were that for very small (n=1 person) efforts where some amount of customization is required, it was easier to quickly modify a client-side tool like Label Me, but Label Studio still feels like the right solution for what you're describing.
*Thread Reply:* @Dan Morris You’re right, I left out some detail. Here’s some things we struggle with on the free version:
• user management — I believe the paid version has this but currently you can’t restrict users to a specific project on the free version. Also, users can’t even reset their password (unless I’m missing something)
• data exploration — there’s no easy way to filter on “images with label x ” ; You can filter on the “Annotations” field but that just gives you the raw JSON for some reason and it’s a pain to work with. Can be crafted as a SQL query but not on the UI. Same goes for more complicated filters like “has labels x and y but not z”
• I’d also love a summary of my data/label distribution (e.g: only 5% of your dataset represents class z)
• We deal a lot with large image files:
◦ For some reason LS takes a lot longer to load/render a large image than it should.
◦ I’d love a way to automatically tile large images and label the tiles then load those tiles into my model (this is something we do manually at the moment)
• We deal a lot with metadata:
◦ Each image has a lot of information behind it that currently sits in a separate database
◦ Related to the dataset distribution I mentioned above, I’d love to be able to plug that data in LS and explore it
• The complexity of LS’s architecture is making us hesitant to fork it and develop against it. For example, I am not sure why the database needs to be so involved but I’m sure there’s a good reason for it.
• I have some complaints about the UI in places but in general it’s OK.
We’re not on the latest version so perhaps some of my gripes have been addressed. The paid version is probably worth it just for the user management but as a small team that only has spikes of labelling throughout the year, the pricing is a bit prohibitive. Having said that, LS Enterprise is probably the cheapest out there from my research.
We’ll never have a perfect tool that has all the features we want, and LS gets close to it, but just wanted to see what else is out there.
*Thread Reply:* This is a really good summary, thanks. This is consistent with the reason that for my own small projects - where I'm the one doing the labeling - I chose Label Me, even after being impressed by Label Studio. The issue I had with LS was primarily the architectural complexity, but it's complex because it's powerful and has all that user management functionality and the ability to handle every imaginable data source and modality, I don't think we can have simplicity and all those things. I also found the metadata management in LS - including just importing existing annotations - to be a bit of a hassle. Well-documented, but a hassle.
Sidebar: this isn't really related to LS, but in terms of running a model with automatic tiling of large images, check out SAHI (https://github.com/obss/sahi).
In any case, thanks for this summary! I don't think any of these problems are unique to conservation, and I'll still come down on the side of re-using off-the-shelf tools rather than re-building. Everyone is likely to have a slightly different set of priorities, but it's good that there's a growing number of stable labeling tools that hit different points on the spectrum from "agile but feature-limited" to "cumbersome but feature-rich" (Label Me and Label Studio are maybe total opposites on that spectrum).
*Thread Reply:* SAHI looks really interesting, thank you for sharing 👍
*Thread Reply:* Deadlines extended!
Submission Deadline: 11:59 pm, April 9, 2024 (Pacific Time) Notification of Decision: April 29, 2024
*Thread Reply:* We will also provide a number of free conference and workshop registrations for the outstanding papers, thanks to our generous sponsors.
*Thread Reply:* Wow this is cool! Check #marine out. Curious to learn more about project, and any specific diving+data communities 🌊
*Thread Reply:* Yup. Already joined it ^_^
I am working with Coastal Impact India where we have set up artificial coral reefs that we are now monitoring using AI. Using instance segmentation algorithms to detect and size up corals.
*Thread Reply:* Hey! Can you share more about your problem? Or even better, that plus a few example images and what you want the classifier to output for each one?
With some prodding from @Dan Morris I'm writing a blog post for label studio on how we 1) cut large geospatial tiles into reasonable chunks that can be read by label-studio, 2) use foundation models to pre-annotate images using the label-studio python sdk, and 3) a couple useful utility functions for converting between geospatial (projected) and image-based (cartesian) coordinate representations of annotations. I would like some feedback on the how to make the explanations clearer. https://github.com/weecology/LabelStudio_BlogPost/blob/main/blogpost.ipynb
<a href="https://github.com/weecology/LabelStudio_BlogPost">weecology/LabelStudio_BlogPost</a>
*Thread Reply:* Let the record show that I didn't just prod and run, I got Ben's tutorial running (proof in the image below!). Will send some minor suggestions via email, but this looks great!
If you need more prodding, I still think there's a Chapter 2 about how to do stuff in Label Studio, then a Chapter 3 where you show all the stuff you do beyond notebook-scale, but I wouldn't recommend actually writing either of those chapters, instead IMO they should be a presentation you give at the next appropriate conference/workshop/AI4RS seminar.
But the ability to do stuff like this is another reason we shouldn't re-invent the wheel; this is some serious power-user Label Studio stuff you're doing here.
Hi! I'm looking into the feasibility of developing a sound recognition model that can detect human and mammalian sounds in Northern Namibia. The idea is to use this model to real-time notify park management about any detections. The goal is to identify (1) a general class for mammalian sounds (i.e., lion, elephant, antelope, hyena, canid, etc.) and (2) a general class for any human sounds (voice, vehicle, music, gunshots, etc.). Any other sounds will be labelled as background (birds, insects, rain, thunder, etc.). Does anyone know of certain datasets, sound-libraries, or repositories where I can find sound files for this project? A semi-extensive search online yielded in the following sources. At this point I'm just looking at global sound datasets, but the preference would be Southern Africa.
If you have any other recommendations or dos and don'ts, please let me know! Thanks in advance :)
Mammal:
Human:
- 550 hunter gunshots: https://zenodo.org/records/5068271
- 850 gunshots: https://www.kaggle.com/datasets/emrahaydemr/gunshot-audio-dataset/data
- 2k gunshots: https://zenodo.org/records/7004819
- 750 gunshots: https://commons.datacite.org/doi.org/10.17632/x48cwz364j.1
- 10k gunshots: http://cadreforensics.com/audio/
- 150 gunshots: https://zenodo.org/records/3519845
- 5TB of urban sounds (cars, motorcycles, music, talking, etc.): https://osf.io/h285u/, https://github.com/brain-bzh/SilentCities
- 9k urban sounds: https://urbansounddataset.weebly.com/urbansound8k.html
- chainsaws: https://zenodo.org/records/5824433
Background (bird):
- 200k birds in several datasets: https://lila.science/otherdatasets#bioacoustics
- 700k birds: https://www.gbif.org/dataset/b1047888-ae52-4179-9dd5-5448ea342a24/metrics
- 3k birds: https://www.gbif.org/dataset/cb6e66f1-3056-404d-a341-bb856762c57c/metrics
- Unknown (website under maintenance): https://www.macaulaylibrary.org/
Background (insect):
- 20K insects: https://www.gbif.org/dataset/ecce3417-d4fd-4fab-9009-09ad3f3f62b3/metrics
- 1k insects: https://www.gbif.org/dataset/bc7c19ec-eefd-403b-931a-16a2ea1e71fc/metrics
- ?k crickets: https://osf.io/5ub4c/
Background (amphibian):
- 1536 frogs: https://www.kaggle.com/datasets/mehmetbayin/anuran-sound-frogs-or-toads-dataset
- 93k frogs: https://github.com/soundclim/anuraset
Background (ambient):
- rain: YouTube e.g.: https://www.youtube.com/watch?v=Mr9T-943BnE
- thunder: YouTube e.g.: https://www.youtube.com/watch?v=T9IJKwEspI8
*Thread Reply:* FSD50K contains strongly labelled human sound events (+ a limited number of animal sound events) https://zenodo.org/records/4060432
*Thread Reply:* You might wish to take a look at PANNs (paper) and inference Python package. We've used this to develop and trial audio monitoring (to detect human sounds) on BBC wildlife productions. There are some links to the Google AudioSet as well, which could be used if you're looking for more raw datasets.
*Thread Reply:* This is fascinating would love to learn more about your work, Peter
Hello I'm new to this Slack, but I wanted to share that I will be running the AI session for the TDWG-SPNHC conference this year if anyone would like to submit a talk (or a talk to any other session). If you have any questions, please feel free to DM me 🙂 :
Hi everyone! Next week I'm presenting a webinar on "Designing Equitable Ocean Technologies," on Thursday March 28 at 3:30PT!
The topic is: new emerging technologies are starting to reshape ocean management and governance: like video/AI monitoring of fishing boats, ocean observation systems incorporating a variety of sensors, and increasingly complex fish stock models. However, we know that technologies have not always had equitable impacts: like uneven Internet/device access (the "digital divide,") and racial/gender biases in AI foundation models. So, how do we ensure that ocean management technology is fair to everyone?
https://oceannexus.org/webinar/
*Thread Reply:* Sounds interesting, will this be recorded? It's at 10:30pm my time 😬
*Thread Reply:* It is recorded! https://www.youtube.com/watch?v=kJRAdUyGHzM
Hi everyone! We (@Jamie Alison, @Quentin Geissmann, and others) are running ACCESS-2024, a funded and selective summer school on computational entomology (primarily imaging & computer vision). It will take place in Aarhus (Denmark) between September 30th to October 4th, 2024, and we are looking for motivated students to apply (deadline April 15th). Accommodation, food and tuition are paid for – selected candidates just need to fund their travel to and from the venue.
Additional info and application details can be found on our website https://darsa.info/ACCESS-2024/. Please circulate to interested people, or repost on X. 🙂
*Thread Reply:* This paper from the Ocean Cleanup might be a useful starting point https://www.mdpi.com/2072-4292/13/17/3401
After reading this amazing paper, which frames the paradigm of application-driven ML (ADML) research—predominantly the type conducted in this Slack—and provides advice on reviewing, hiring, and teaching, I'm left wondering how you would like, or what you see as necessary, for your conservationist collaborators to improve dataset creation, task framing, and criteria for success... I believe many of us have encountered claims of having a well-annotated dataset, only to find out it wasn't quite so. But at the same time, sometimes weak labeling can be applied to significantly reduce the workload. Interdisciplinary collaboration is very necessary, so are you aware of any guidelines or resources that guide conservationists to better collaborate with ML professionals? Do you think it could be a useful? How do you usually do it?
The closest thing I'm familiar with is the guideline provided by @Dan Morris in his talk at CV4Ecology 2023.
*Thread Reply:* Hi Anton, thank you for sharing that excellent paper. I am thinking along the same lines, having done a similar exploration with astronomers https://arxiv.org/pdf/2211.14401.pdf and now working with conservationists. Happy to discuss!
*Thread Reply:* Non-expert here, setting up our first ‘applications-driven’ computer vision project. These were helpful to help me reach out to our CV-experts that we’re going to be working with 🙂 thanks for sharing!
*Thread Reply:* Hi @Nina Grace Baranduin I'm working with a group here at Ohio State studying the African Wild Dog pack at the The Wilds here in Ohio. We are gathering data with camera traps and drones.
*Thread Reply:* check out the folks at WildCru! And African Wildlife Conservation Fund. Zambian Carnivore Programme too
*Thread Reply:* Hi @Nina Grace Baranduin! Also a group including me at Cornell studying dholes! We're working with bioacoustic data!
*Thread Reply:* I believe the Abrahms lab at UW (https://www.abrahmslab.com/) may also be doing some of this with wild dogs?
*Thread Reply:* Whilst not a review, this paper used GPS tracks to classify bull elephant behaviour into Musth/Non-Musth state via HMMs (state space models) 🐘 https://www.savetheelephants.org/wp-content/uploads/2019/06/2019_Taylor_et_al-2019JAE_reproductivetactictsofbulls.pdf
Hi everyone! Wildlife Protection Solutions is hiring a full-stack software developer (Python/React) with a focus on AI/ML for wildlife conservation. See the attached doc, the post on WildLabs, or contact info@WildlifeProtectionSolutions.org for more details about the position. Thanks!
*Thread Reply:* If MegaDetector is working well and you want to add species classes, 99% of the time I would advise you not to fine-tune MegaDetector. Instead, consider training a separate classifier that operates on the pixels inside the MegaDetector boxes. MEWC offers a good step-by-step process for doing this:
https://github.com/zaandahl/mewc
The only cases where I would advise fine-tuning MegaDetector are the cases where MD itself isn't working well on your species, but this is a more substantial process, and you'll need to come up with bounding boxes somehow, and 3000 images is unlikely to be enough. But, FWIW, if you want to venture into the unknown, I've done this a couple times, specifically for those cases where MD struggles (in both cases with reptiles):
https://github.com/agentmorris/unsw-goannas https://github.com/agentmorris/usgs-tegus
Those repos give some instructions about exactly what I did, but it's a pain!
*Thread Reply:* Clarifying: the "fine-tuning MD" part is easy, not because there's anything special about MD, but because fine-tuning YOLOv5 is easy. It doesn't even require any code, the YOLOv5 CLI is amazing. But creating a dataset of bounding boxes is a PITA, so I wouldn't do that unless you need to.
I'm starting to play around with overhead images that lack georeferencing where we want to estimate the unique number of individuals per species in a set of imagery. Consider these two images captured from a piloted flight past a bird colony and a hand-held camera. Human reviewers can use the background textures to try to estimate overlapping areas, and then manually decide whether to count the birds in that portion of the photo. Its fairly trivial using old-school computer vision to stitch them (see example with matching keypoints), but you can easily imagine that stitching together a large mosaic will naturally warp the images, especially as the plane turns, making downstream machine learning difficult. Instead of stitching then predicting, i'm thinking about this as a 're-id' problem, except for the fact that my objects are not visually different, like whales or tigers. By using the background context, we should be able to match up duplicate detections, assuming the photos are taken in rapid succession. We can then use more general deep learning methods like (https://openaccess.thecvf.com/content/ICCV2021W/DSC/papers/BansalWhereDidISeeItOb[…]anceRe-IdentificationWithAttentionICCVW2021paper.pdf). My idea is to use our existing object detection models, which are pretty good, and then search all corresponding images for matching background context. I know others are thinking about this space (perhaps @Lasha Otarashvili, I can see a annotation tool in Scout), all thoughts welcome. I am working with my annotation team to curate a dataset, but I sense there are others out there already. Helpful data would be more airborne and fuzzy objects, and less tiger stripe camera trap data. They feel like different problems. I think this problem of unordered, non-curated sequences of airborne photos could be useful to a large monitoring audience (e.g African Savannah herds @Ben Koger, @Howard L Frederick, Coral Reefs, and other large aggregations)
<a href="https://github.com/OpenStitching/stitching">OpenStitching/stitching</a>
*Thread Reply:* Oooh this is one of those fun threads where we can place our bets on what will actually work, then you'll tell us how wrong we were. I see three approaches here: (1) stitch then detect (using something like SAHI to divide inference into patches), (2) detect on the original images, register the images and translate the detections accordingly (without explicitly producing a mosaic), and use basically-off-the-shelf NMS to remove redundant detections, and (3) the fancy stuff you're suggesting where you explicitly search for matching context.
My bet is that object detection will be pretty robust to warping, as long as you train it on a representative distribution of warping artifact, so my bet is on (1) being the best bang for your buck, followed by (2), and I think (3) will also work, but will get the same results for lots more code.
If you have to train on data that won't see a lot of distortion from the registration, but you expect a lot of distortion at inference time, then my bet switches from (1) to (2).
I've placed my bets!
*Thread Reply:* I would personally start with some off the shelf multi-object tracking approaches. Assuming you have high-enough frame rates, these should generally work. Something like DeepSORT or whatever the current SOTA variant of that is. I do like the idea of adding a little bit of context though around each detection to help with the matching portions.
*Thread Reply:* I see, I guess a potential issue here is the constraint of working with un-ordered data. Though assuming you had a raw video, this shouldn't really be true? Also if you are assuming you have enough overlap, then stitching as you suggested (or fancy things like Metashape or COLMAP or something) should get you an ordering, and then you can leverage existing MOT pipelines.
*Thread Reply:* not raw video, just hand held camera photos. think a series of 10-20 photographs.
*Thread Reply:* Ah I see, so not at all high framerate. My initial thought is (1) and (2) are probably a bit more blended. I think with (2), you could align them by stitching, but only to come up with an "ordering" and treating as a sequence, and then use MOT-style matching with inflated detection regions, you might get some nice robustness properties. With (1) I'm not sure how you would handle the actual stitch without running a detector over the individual images first anyway?
I would be curious if you're able to just flat out take the inflated detections and do a DeepSORT ranking and thresholding and just see if they match up (maybe with a bunch of discrete rotations). I think this is already fairly similar to your proposed approach, but just comparing detections between all images, rather than searching through the whole image?
*Thread Reply:* Interesting. It sounds like a special case of multi-view object segmentation
*Thread Reply:* This is assuming the animals don't move at all?
*Thread Reply:* I think they are allowed to move in small areas, but obviously not in major ways. The images are usually captured in rapid sequence, like through an airborne transect as a human photographer passes. I'll keep sharing in this thread as I proceed, thanks everyone for feedback.
*Thread Reply:* I could imagine that this would work better for ecosystems with some types of background texture than others, very curious to hear how things go!
*Thread Reply:* @Ben Weinstein this is indeed exactly the kind of thing we are interested in for the Scout annotation system (and others). I think circling a large colony or herd is a different problem but figuring out adjacency and image footprint all help solve the same problem:
- We often need to de-duplicate observations in adjacent images along flight path.
- Sometimes we need to just track who’s been seen in one image and figure out which members of a herd were occluded by vegetation (or neighbours) in the adjacent images.
- We typically at a minimum have ordered images from the aircraft (sometimes with GPS and IMU data), and usually can be confident of 40-50% overlap between image pairs.
*Thread Reply:* Matching adjacent oblique images with keypoints from a moving platform seems to work really poorly with SIFT / SURF - but @Hannes Naude (Innoventix, South Africa) has a tool for footprint matching between adjacent obliques that uses ‘lightglue’ and ‘superglue’ - no need for IMU data, just checks timestamps or GPS to confirm a given pair is worth trying to match.
*Thread Reply:* Nifty little demo of matching available here: https://huggingface.co/spaces/Realcat/image-matching-webui
@Ben Weinstein trying the superglue match with superglue / RANSAC on your 2 images gives 155 matches.
*Thread Reply:* Hi all.
Thanks for pulling me in to this fascinating topic @Howard L Frederick.
Yes, this sounds like a problem that might be handled well by the registration pipeline we have in Detweb (machine guided annotation and analysis tool that we originally built for ESS, but that we are currently trying to open up to more general use). It uses approach (2) in Dan's list, which works well for game surveys (both oblique and Nadir views) in the absence of stationary content in the images (visible wings, struts, wands etc. cause it to fail pretty quickly, We expect this to be fixable with some basic masking, but that is not implemented yet). I'm a little worried about scaling for the number of sightings you need to match in a dense bird colony (we use optimal assignment aka Munkres algorithm internally which scales as O(N^2)), but am keen to give it a go. Can you share a handful of representative images somewhere? I can pull it into the system and let you know how well it works or how badly it fails.
*Thread Reply:* OK, So I quickly started a new project in Detweb, uploaded your images and (manually) annotated the birds I saw. I then launched an image registration task and it seems to handle the matching quite well.
I include a little screencast that shows how this works.
*Thread Reply:* @Dan Morris I’m betting #2 on your list.
*Thread Reply:* Awesome, where can I read more about the tool/strategy, is it possible for me to extract the relevant part of that code, or atleast get a sense for the general steps. I can generate a few more image pairs today. I am working on an annotation workflow for my team to generate eval data.
*Thread Reply:* Thanks @Howard L Frederick, I made a discussion on that repo, let's see what the maintainer says. https://github.com/Vincentqyw/image-matching-webui/discussions/28. Do you have sample overlapping images with annotations that we can add to an eval set? just one or two.
<a href="https://github.com/Vincentqyw/image-matching-webui">Vincentqyw/image-matching-webui</a>
*Thread Reply:* @Ben Weinstein Sure, I've been promising for years now to open source the code, but I haven't gotten so far (mostly because it doesn't live up to my own standards for how clean open sourced code for general consumption should be and adding features has always outranked doing the long overdue cleanup). Last night I decided to just take the leap so the uncleaned code has been pushed to github.
So the problem of re-identifying multiple sightings as corresponding to the same individual is broken up into two parts:
- Figuring out how the two (for present purposes) images relate to one another. We assume that we are dealing with aerial images from a significant height so this can be well approximated with a Homography.
- Given the relation between images figuring out which sightings are paired with which and which sightings are unpaired (either because an animal was spotted in image A and missed in image B, or because an incorrect sighting was made in image A which does not match to anything in image B) The user will then typically need to manually resolve the second category. Detweb tries to do as much as possible directly in the frontend (written in React) and just push results to serverless services on AWS (DynamoDB mostly). However part 1 above requires CUDA so it is implemented as a ECS container image with all the dependencies pre-installed, which gets launched automatically when the user pushes image registration work to a SQS queue. The relevant code is in https://github.com/WildEyeConservation/Detweb/blob/develop/cdk/containerImages/lightGlueImage/code/processSQS.py and uses LoFTR to do the matching. A much cleaner example (without all kinds of AWS integration and housekeeping clutter) is in https://github.com/WildEyeConservation/Detweb/blob/develop/cdk/containerImages/lightGlueImage/code/lightglueMatch.py You seem to be familiar with traditional keypoint matching, so there will be nothing new here for you. The only change is to use SuperPoint (rather than SIFT/SURF) for keypoint detection/description and LightGlue for matching. From that point on you can use the matched points for whatever (typically throw them into RANSAC to estimate a homography).
The most relevant code for part 2 lives in the React frontend, namely https://github.com/WildEyeConservation/Detweb/blob/develop/src/useOptimalAssignment.jsx Here I use the Munkres algorithm to calculate an optimal assignment between pairs and create a new set of annotations that are displayed to the user for confirmation. This code is currently being actively worked on and is not really commented. Most of the complexity here has very little to do with the algorithms and mostly with a self-taught React developer wrangling React to try and get an acceptable UX. 😳
I'd be happy to answer any questions you may have.
*Thread Reply:* Sure, i'll respond in DM not to clutter thread.
*Thread Reply:* For those interested in following this thread, here is where I am in terms of a reproducible example. https://github.com/weecology/DoubleCounting
*Thread Reply:* And with an image. This is using simple serial homography among images as described by @Hannes Naude. I still want to explore the 3d approach since it makes fewer assumptions, but for far away images and rapid sequence, this is a reasonable start. I will be wrapping this into DeepForest for users to remove double counts among a series of images.
*Thread Reply:* If others here have sample data I can run through this, i'd like to write up a blog post. @Howard L Frederick
*Thread Reply:* For those following along on me trying to reduce double counting in non-uniformly overlapping airborne photos combined with object detection, here is the first example of the success. In blue are original detections that are deemed doubles, pink are detections that are maintained. The selection rule here is 'left-hand' meaning that when two detections overlap, we choose the earlier image. I have also written 'right-hand' and 'highest score'. I am starting to doc this if it useful to others, it should only be considered a simple solution and serve as a baseline for more sophisticated approaches. https://github.com/weecology/DoubleCounting
*Thread Reply:* https://arxiv.org/abs/2106.04972 https://towardsdatascience.com/how-to-use-confidence-scores-in-machine-learning-models-abe9773306fa https://towardsdatascience.com/aleatoric-and-epistemic-uncertainty-in-deep-learning-77e5c51f9423 https://blog.paperspace.com/aleatoric-and-epistemic-uncertainty-in-machine-learning/ https://arxiv.org/abs/1910.09457
*Thread Reply:* If you're looking for a general tutorial, Eric's links look great. If you're looking for something specific to conservation, I don't know of anything that's really a tutorial, but this paper is about confidence scores in the context of camera traps, so the introduction maybe isn't far from what you're looking for:
https://www.biorxiv.org/content/10.1101/2023.11.10.566512v2.full
*Thread Reply:* Hey Ghazi - I’m sure that others can chime in with better examples, but I know of a few teams that fulfill at least 2 out of the 3 dimensions (AI/DS + Conservation/Environment in particular).
- Lea Berrang-Ford’s collaboration
- A multi-institution food systems project led by Jaron Porciello
- A project on mapping climate impacts from the Mercator institute
- At the risk of tooting my own horn, I’ve co-led a project with The Nature Conservancy using LLMs to categorize and map evidence for natural climate solutions and their impacts on people and nature. The first paper in this body of work is currently under review. All of these projects largely focus on using NLP (and LLMs / transformer models) in particular to examine published (and sometimes gray lit) datasets at a large scale. I haven’t seen research groups working across AI + conservation + law in particular, but I’d be curious to hear about examples.
https://x.com/lauransotomayor/status/1781739493579522500
> Today, Meta and World Resources Institute are launching a global map of tree canopy height at a 1-meter resolution, allowing the detection of single trees at a global scale. In an effort to advance open source forest monitoring, all canopy height data and artificial intelligence models are free and publicly available. https://sustainability.fb.com/blog/2024/04/22/using-artificial-intelligence-to-map-the-earths-forests/
*Thread Reply:* This is huge! The DinoV2 backbone is super powerful and having one that’s pre-trained on satellite and aerial data is amazing. It will certainly generalize well beyond trees.
*Thread Reply:* @Nate Harada do you want to show a fine-tuning example? We could rig up some bird data from https://zenodo.org/records/5033174
*Thread Reply:* yeah was gonna try and play with this when I have some time
*Thread Reply:* Quite interested in this proposed use case above. What would the outputs of this proposed fine tuning look like?
*Thread Reply:* Thank you for sharing Carly! I am also curious in working with fine-tuned applications!
*Thread Reply:* Hmm so I’m seeing a developing opinion that the actual tree height map isn’t that good, curious if people who actually need to do canopy height estimation have opinions on that specific task. I’m looking more specifically at using the backbone only as a starting point for other tasks, for example few-shot classification for wildlife or tree species detection.
*Thread Reply:* @Nate Harada could you link to places you’ve seen those opinions? or is just in-passing type convos you’ve heard
*Thread Reply:* Some was word of mouth, but a few tweets (obv take w a grain of salt cause some are competitors):
https://twitter.com/AdPscual/status/1783515862365831623 https://twitter.com/TC_Chakraborty/status/1783839579457614165 https://twitter.com/arjenvrielink/status/1783795483585970442
I have no skin in the game for this particular application, just sharing what I’ve seen/heard. Would be interested to hear people finding the model useful for their problems! I’d love if the backbone was broadly useful, personally.
*Thread Reply:* interesting feedback!
Hey everyone🎊 I'm glad to be a part of this community! I am a Ph.D. student affiliated with the Vision Lab at IIIT Delhi, India. I just realized that this might be a good space to showcase our recent work on AI for Agriculture, recently accepted at IEEE WACV 2024 as an oral presentation.
SICKLE: A Multi-Sensor Satellite Imagery Dataset Annotated with Multiple Key Cropping Parameters 🌾 What is SICKLE? 📡 It is a first-of-its-kind dataset constituting a time-series of multi-resolution imagery from 3 distinct satellites: Landsat-8, Sentinel-1 and Sentinel-2. Our dataset constitutes multi-spectral, thermal and microwave sensors during the January 2018 − March 2021 period in the Cauvery Delta region of Tamil Nadu, India.
🌾 What are the key features of the dataset? 📡
- The dataset consists of images from multiple satellites having a variety of sensors.
- Each agricultural plot is annotated with multiple cropping parameters, for example, crop type, crop yield and phenology dates, also allowing multi-task learning.
- The annotations are available at 3 scales, i.e. 30m, 10m and 3m, also allowing high-resolution inference from low-resolution satellite data.
- The dataset is organized in a way that can be readily used by researchers from multiple domains, including agronomy, remote sensing and machine learning.
- Each temporal sequence in the dataset is constructed by considering the cropping practices followed by farmers primarily engaged in paddy cultivation.
- Tasks that can be performed: semantic segmentation, crop phenology date prediction, yield prediction, panoptic segmentation, synthetic band generation, image super-resolution, cross-satellite sensor fusion, etc. 🌾 Why is this dataset important? 📡
- Monitoring the cropping pattern is a composition of multiple tasks, for instance, crop classification followed by yield estimation. SICKLE is the first work to provide annotations of multiple cropping parameters for the same set of plots.
- Heterogeneous farming is followed in the study region having different growing seasons. Moreover, the primary crop (i.e. paddy) is grown 2-3 times in a year. Thus, the straight-forward way to use a time-series of satellite images (say all images from Jan-Dec) is not suitable. We present a novel strategy for preparing time-series data over a seasonal temporal window that is consistent with the regional cropping standards that are typically followed by the farmers for crop production.
- Majority of the plots in this study region are small farms, with more than 95% plots having an area of less than or equal to 1 acre and an average size of 0.38 acres, thus making prediction tasks more challenging for low and medium resolution satellite images. In case of any queries, please reach out to me at depanshus@iiitd.ac.in. Also checkout our short YouTube video to get a high-level overview of our work. If you think this work might be beneficial for you, please visit our official website.
*Thread Reply:* @Hugo Magaldi I am also new to the conservation area, so I'll give my opinion from the perspective of a core-ML/CV person.
- You can try out focal loss instead of cross-entropy loss. The focal loss can simply be thought of as a variation to the cross-entropy loss, where a higher weightage is given to the samples that are hard to classify (in this case, the samples that are under-represented). It is a pretty standard and easy approach to deal with class-imbalance.
- I am not sure if I understood the term pseudo-duplicate correctly. This might be because of my lack of awareness about the FiftyOne. But I am assuming that by pseudo-duplicate you mean that the same image is replicated in the dataset multiple times but with slight variations, such as cropping or rotating the image. If this is the case, I'd suggest you to first analyze the dataset. If you find that the class imbalance is just because of these pseudo-duplicates and without these duplicates the dataset would have been somewhat balanced, then you might not even have to employ the focal loss. Instead, you can implement a class-specific augmentation technique. By this I mean that for the under-represented classes in your dataset, you can virtually add more samples for such classes by applying random augmentations. A simple example of this could be that if you have two classes in your dataset, say X and Y, and only because of these pseudo-duplicates 80% of the samples in the dataset belong to X. Then in such a case, you can achieve a balanced dataset by randomly cropping, rotating and flipping the images that belong to class Y, such that the two classes have the same proportion of images. I hope this helps. Let me know in case you have any further questions.
*Thread Reply:* Re: pseudo-duplicates... do you mean images from the same sequence/burst that are just similar, i.e. where an animal hasn't moved between images, or literal image copies? If you mean images from the same sequence/burst where the animal hasn't moved, my recommendation would be to leave them in, unless you're really limited by training cycles.
Re: rare classes... I don't have a very useful technical answer; class weighting seems reasonable to me, and more machine-learning-y people than me can weigh in on weighting schemes and loss functions. But I'll give an unsolicited non-technical answer: I think sometimes we worry too much about performance on rare classes in a way that is detrimental to overall system performance. If you are training a real-time classifier whose only job is to report the instance of rare classes, by all means, weight those heavily. Similarly, if you have your sights set on full automation, that's... ambitious, but then yes, you need to perform well on rare classes and it might be worth paying a price in accuracy on common classes.
But for the more common scenario where you're training a classifier to help someone process a gazillion camera trap images, more often than not the benefit you provide to the user will be entirely dominated by having high precision and adequate recall on the most common classes, even if their focal species are rare, and the marginal cost of having the user review the rare classes manually is very small. I'm not telling you not to focus on rare class performance, just reminding you to consider the use case and to evaluate whether hurting performance on common classes is worth the lift to performance on rare classes (there's no free lunch!).
*Thread Reply:* @Depanshu Sani Thank you for your answer. I will try out focal loss. By pseudo duplicates I meant very similar images coming from the same sequence. But indeed data augmentation can be useful for underrepresented classes.
*Thread Reply:* @Dan Morris Thank you for your insights. Taking into account the time for user review, much smaller for rare classes, is a good point. The example I had in mind is a classifier for 5 species of monkeys, with 1 specie making up for 90% of the dataset. The other 4 are not rare species, just less observed, and I still aim for a good accuracy on all 5 classes.
*Thread Reply:* Also, in case anyone was wondering how MegaDetector does on zombies, here are MD results for more or less the first hit for "zombie apocalypse" on Google images:
https://twitter.com/sarameghanbeery/status/1785382847135760472
*Thread Reply:* For those that don't use twitter:
Online citizen science platforms like iNaturalist and Macaulay Library contain a wealth of images but are hard to search using text. We are looking for ideas so we can develop the next generation of AI models that can help users to better search these image collections.
We are currently building a new dataset that consists of wildlife images combined with text descriptions. We want this dataset to be representative of the real world interesting questions that citizen scientists, ecologists, researchers, etc. want to answer.
As we get started, we hope to collect ideas from the community about what types of image searches they would be interested to perform. If this is of interest, please fill out this short form: https://forms.gle/CmRf826AkJPkvhjX7
PS You can fill it out more than once.
FathomVerse is now available to download for free on the App Store and Google Play!
FathomVerse is a cozy, community science experience that will take you on an amazing journey through the depths of the ocean. Not only that, it also helps improve AI models that scientists use to discover ocean life. FathomVerse allows anyone with a smartphone or tablet to take part in ocean exploration and discovery.
• 🎮 Play minigames to interact with real images collected by researchers and discover ocean animals. • 🔎 Hone your skills and learn how to visually identify 40+ groups of ocean animals. • ❣️Save your favorite images and curate a personal gallery. Join us on an ambitious journey to create a game that connects the power of community with cutting-edge technology for the benefit of ocean life. Dive deeper into the FathomVerse community on Instagram, TikTok, and join our Discord.
Thanks to lots of members of the AI for Conservation community, including @Sara Beery @Oisin Mac Aodha @gvanhorn @Tanya Berger-Wolf @Genevieve Patterson for input and support.
Let us know what you think!
Onwards and downwards!
*Thread Reply:* Our team uses git-lfs for test cases for computer vision use cases because I like the github repo being the single source of truth for a given project. We store our data on GCP typically (in buckets) — and we do version control that (Google will do that for you albeit for an additional cost). For metadata/annotations we either release via a bucket, or on GH directly if they are small enough. We have also looked at (but don't currently use) dvc.org which we found was not optimal at the time for larger datasets (TB/PB), but that may have changed since.
One useful abstraction that is worth thinking about is decoupling the human/machine annotations of your data from the raw data. In our cases, we get raw data from a lot of public sources (like NASA/earth data/ESA), and it may be most convenient for users to access the raw data from those APIs directly (although not always)
*Thread Reply:* We use Hugging Face as a model and dataset repository and archive (they partnered with DataCite to provide DOIs). Larger files are stored through git-LFS (at no cost), and it has a lot of nice integration features through huggingface_hub for loading models and datasets, and even running demos of various models and projects. This also makes it easier for integration with code in GitHub; you can also load assets through URLs. The UI isn't quite as nice as GitHub for active development (PRs and branches have less clear, easily accessible features since they're merged into one), but overall it's been working well for us.
Hi everyone! Has anyone implemented DeepFaune's classification model in their own project? I'm currently trying to use their model for a automated camera trap-based alert system for bears and wild boar in Romania. It would be extremely helpful to see how others have managed to get the pre-trained models to work in their own pipeline, if anyone is willing to share their code!
Ref: https://plmlab.math.cnrs.fr/deepfaune/software/-/tree/master
*Thread Reply:* @Piotr Tynecki - any insights in addition to your posts hugely appreciated! https://wildlabs.net/discussion/successfully-integrated-deepfaune-video-alerting-system
*Thread Reply:* @Vincent Miele CNRS
*Thread Reply:* Are you working with @Thijs and Hack the Planet?
*Thread Reply:* @Tom Ratsakatika I will share the post update this week.
*Thread Reply:* @Tom Ratsakatika did you saw this reply? I shared the code snippet in PM.
*Thread Reply:* For drone videos, we worked on software for video annotation. It has an inbuilt detector for initial detection and then uses a tracker to track the animals across consecutive images. ( https://github.com/robot-perception-group/smarter-labelme / http://doi.org/10.1007/978-3-031-44981-9_12 ) maybe that could help in your case. It comes pre-trained with the MS-COCO object detection classes but once you have a few hundred images annotated you can re-train the detector including those, which should make it much better.
*Thread Reply:* don't hesitate to send me a message if you have questions about installation or usage
*Thread Reply:* Hi @Peter van Lunteren, we are actually also working on such kind of things within our research project BAMBI (https://www.bambi.eco/). Currently, we are still finishing the labelling work. But within the next months we plan to release some animal detection models trained on some thousands of individual animals, visible in ten of thousands of video frames 🙂 Maybe there is some way of collaboration there 🙂
*Thread Reply:* The team at WildMe has been working on this as well, @Jason Holmberg (Wild Me)
*Thread Reply:* @Peter van Lunteren you can train multi-class models easily through our API as well. A general 'animal' detector is something we have discussed, but not taken up. https://deepforest.readthedocs.io/en/latest/. We have found starting from our 'bird' model you can train to other animal like deer in just a few annotations.
*Thread Reply:* Thanks for the tips - this is great! I'll play around with the options and get in touch if I have specific questions. This slack channel is very valuable 🙂
Yesterday the special issue on automated insect monitoring came out in Philosophical Transactions of the Royal Society. There are papers on computer vision, acoustic monitoring, radar and molecular methods. It also has a lot of opinion pieces and perspective pieces that are relevant to all of these technologies. For example dealing with the needs of insect monitoring in low and middle income countries, data sharing, citizen science and long term monitoring. https://royalsocietypublishing.org/toc/rstb/current
Roel van Klink is compiling a folder with all PDF's. Feel free to download everything here, and share the link: https://www.dropbox.com/scl/fo/zjsc6oyghabhs5lngl14o/AK9MFw0GQawiMRDKYVa65Fs?rlkey=b5uofp8ogg37og2nc0jwofty1&dl=0 [text taken from a post by Roel in another Slack workspace]
*Thread Reply:* And in a bizarre confluence of universes, Molly's father taught me everything I know about op-amps (which still isn't a lot, but whatever I know, it's thanks to Molly's dad).
*Thread Reply:* this might be of interest - https://biodt.eu/
*Thread Reply:* Take a look at this as well.
Brian Mayton gave our team of the tour and sensor network, and it's pretty impressive stuff! Possibly more audio-oriented than your computer vision focus, but maybe still of interest?
*Thread Reply:* Thank you both, fascinating projects! Any other recommendations welcomed
*Thread Reply:* https://www.frontiersin.org/articles/10.3389/fmars.2023.1200408/full, but also, https://arxiv.org/pdf/2207.09295 @Justin Kay
*Thread Reply:* This is great, thank you!
*Thread Reply:* One of the graduate students at my school, Christian Swartzbaugh (css36162@uga.edu), is using AI to count and track individuals of two different fish species in experimental tanks. He's advised by Stacy Lance (https://www.lancelab.org/). I'm sure you could reach out to either of them. Attached is the abstract from our annual symposium where he presented on it.
*Thread Reply:* I‘m involved in a similar project primarily counting river herring, led by Robert Vincent at MIT (rvincent@mit.edu). Feel free to reach out!
*Thread Reply:* [shamelessly pasting my reply to a similar message on the help_needed channel]
I have been maintaining a list of all the publicly-available models for finding fish in imagery that I'm aware of:
@Filippo Varini and I have also been keeping track of public datasets suitable for training models like this:
https://github.com/filippovarini/filippo_public/blob/master/fish-datasets.md
*Thread Reply:* Thanks Timm, I would appreciate that. Should I reach out to you or Robert?
*Thread Reply:* @Gerald Maschmann feel free to reach out to Robert first, he knows best who to include in the conversation
*Thread Reply:* Hi @Gerald Maschmann, we started the work on a full stack analysis pipeline for the project described in Frontiers by Atlas et al. Happy to chat what we are doing ed.
*Thread Reply:* Another option is VIAME which is a toolkit that's used by a number of the NOAA fisheries offices. I was briefly involved in the development a few years back and am happy to answer basic questions or connect you with the people still working on it at Kitware.
*Thread Reply:* One important features is it's designed specifically to make it easy to train models for your own tasks.
*Thread Reply:* Ok, thank you! This is great. I will definitely be in touch. Right now I'm in the middle of planning for the salmon season, so it might be later this summer or even this fall before I circle back.
*Thread Reply:* Hi @Gerald Maschmann, I advice you to also check out the new MBARI open source models on hugging face. Here.
You can play around with a few examples in this interactive dashboard
*Thread Reply:* Here is some recent work that focuses on detecting fish in sonar imagery using AI, and improving performance at new locations you haven't trained on. The authors are very knowledgeable and have been working on salmon detection. Might be a good idea to reach out to them.
https://arxiv.org/abs/2207.09295
*Thread Reply:* @Gerald Maschmann check out fishial.ai there a demo at https://portal.fishial.ai/search/by-fishial-recognition the system can be used to collect and label image of fish. The segmentation model is spot on.. If you want to talk email support@fishial.ai
*Thread Reply:* Incidentally, on a side note, I've had a play with chatGPT4o and the initial results looks pretty impressive. I asked it how it was getting the results and it said (if you can trust it) that it was making use of the text description that you might see in a spotter's guide. Which is an interesting approach for making performance of AI results more human understandable.
*Thread Reply:* @Zhongqi Miao has been looking at this recently 🙂
*Thread Reply:* Hi hi! Is there any limitation/constraint you would need to take into account? (model size, inference speed,...)
Nowadays, I would say ResNet-like architectures are good enough 👍:skintone2:.
Nevertheless, in my personal opinion, good performance comes down to other factors such as: good quality data, loss function choice and a suitable data pipeline, among others.💡
*Thread Reply:* Suggest a pretrained transformer like DinoV2 or SigLIP unless you have a ton of labeled data or strict latency requirements
*Thread Reply:* Data choices still almost always matter more than architecture
*Thread Reply:* @Robert Dawes You can train and benchmark YOLO-based architectures (YOLOv8, YOLOv9, YOLO-NAT, etc.) with Vision Transformers (ViT) like RT-DETR, or try a novel approach based on Visual Language Models, such as CogVLM or Large Multimodal Models like PaliGemma.
All of these approaches will provide interesting and accurate results for your issue, but as @Sara Beery said, the key to your focus should be the gold standard dataset and smart + eco-based data stratification before launching fine-tuning stage.
*Thread Reply:* Interesting to hear about these transformer based approached (which of course didn't exist when we created our original pipeline)
*Thread Reply:* Have you got any tips to ensure our data is good as possible? Or any suggest papers that have followed a good approach?
*Thread Reply:* And in answer to the performance issues, we previously used Inception v3 because it was very quick and cheap - and didn't require much compute. This time round we have scope to be a bit more flexible.
*Thread Reply:* It really depends on how you define "better". I personally is still using resnet18 for most of my classification projects because i don't need the extra 2% to 5% improvement from the architectures. And the majority issues are from data itself like imbalance and noise, which can't be solved by model architectures. Ultimately, we will rely on human in the loop, so as long as we have a good calibration pipeline, it is applicable. So what is in your mind that can make your current pipeline better? That Would be a good starting point.
*Thread Reply:* @Robert Dawes One good tip is to collect, store and process the data in Camtrap DP standard (paper), especially if you have data from many sources from many research groups.
*Thread Reply:* @Zhongqi Miao - our current pipeline has classifiers for a couple of domains which we've been using for a few years. We've now got some work that would involve expanding to look at other domains/locations and could also bring in some new training sets. So we felt that if we're going to do some more training we ought to try and at least think about what model we're training.
What "better" means could actually be open for debate. It might be just better classification, but it could also mean being quicker to retrain with new data or being as cheap as possible.
*Thread Reply:* At the very least we should compare something like resnet to our current model to see what improvements or issues there are
*Thread Reply:* @Robert Dawes I see! I think in your case, it is more of a domain generalization task. First of all, in terms of a place to check new models, https://paperswithcode.com/sota/image-classification-on-imagenet, this leaderboard is always a good place to start, and you will see almost all models on the leaderboard are transformer models with huge parameters and huge pretraining or some contrastive pretrainings. Transformers are also notoriously hard to fine-tune. So I don't know whether these model will be useful for your project. However, these models, because they are trained on large data, are usually good at domain generalization on general domain data. But for camera traps, I have doubts, because the domains are so different from general images. You can still see resnet-152 on the leaderboard though.
*Thread Reply:* On the other hand, since it is a domain generalization tasks, I think searching for domain generalization targeted methods might be a more effective way. Even collecting enough training data from different domain can be an option I think
*Thread Reply:* I’ve wondered in the past how good a benchmark ImageNet is for wildlife classification given that most objects in there don’t have the characteristics of wildlife - deformation etc
*Thread Reply:* I - like most people - have fine tuned networks trained on ImageNet. But I wonder if that has some flaws as an approach.
*Thread Reply:* Thanks very much for the notes above
*Thread Reply:* Exactly. Therefore those leaderboard is only a place for people to know what's the state of the art classification models are. And usually, as long as finetuned well, better models on the leaderboader can yield better results on animals. But the tricky part is "finetuned well". And like you said, imagenet pretrained models And wildlife imagery have domain discrepancies. Therefore out of the box, no imagenet models work on camera trap images. However, all the low level visual features are the same, they are all edges, textures, lines, And shapes. Imagenet is good because it covers most of these low level features. And finetuning is basically a way to transfer the combinations of these lowlevel feature to other domains like wildlife imagery. And this is also why tramlnsformers trained on huge pretraining data work even better because these data cover more lowlevel features and possible combinations
*Thread Reply:* Oh wow this is interesting.
Would you recommend going to the event as a "private person"?
*Thread Reply:* Which of the evenings are you looking at? I might be coming. Greetings from Austria, Robin
*Thread Reply:* Yay Robin! We're thinking a social following the Apero event on the Monday evening and a meeting Wednesday around/after lunch, but still finalizing
*Thread Reply:* Lightning’s modularization is honestly really nice. I know other frameworks (i.e. lightly) include tutorials and documentation in both pytorch and pytorch+lightning, which is definitely more work from a documentation perspective, but is also really nice from a user perspective to not be forced to use one or the other.
I know @Markus Marks has thoughts on this too
*Thread Reply:* I have worked with both plain PyTorch and PyTorch Lightning in different projects. PyTorch Lightning offers many great features such as automatic device handling, support for various metrics logging platforms, scaling of training across multiple devices, early stopping, checkpointing, etc. In my experience, this speeds up development, especially at the start of a project. It also helps to increase the maintainability of the code, as you have to write, test and maintain less of the code yourself. However, the learning curve for PyTorch Lightning is also steeper. For people who have little developer experience with deep learning, I would therefore recommend gaining experience with plain PyTorch first. For me, this helped to understand what happens at which point in a training or inference pipeline. Another disadvantage of PyTorch Lightning is that you have less control over the training and inference pipeline, at least if you are working with the standard Lightning Module. For projects with many non-standard steps in training or inference, plain PyTorch is the better choice in my opinion.
*Thread Reply:* We use lightning for DeepForest and its really nice to package together a single class, it allows us to store a config file
m = main.deepforest(<you can put a config file here>)
and then be able to do things like
m.create_trainer()
the only awkward piece is sometimes a pytorch lightning trainer wants the module specified explicitly, so it can look alittle silly.
m.trainer.fit(model=m)
but otherwise it really saves the user from needing to know things and would allow you to swap out models and architecture in the future without the user needing to care about it. If your goal is to make user experience easier, I would use lightning, if your goal is to give maximum flexibility for just your team, plain pytorch. Here is the example of our lightning subclass https://github.com/weecology/DeepForest/blob/7de55c1a941ab53dc06158cb84f8fd3ca5ca5b0a/deepforest/main.py#L22
*Thread Reply:* @Ben Weinstein this example reminds me of something I’ve wondered with PyTorch Lightning - the API requires a separate trainer object from the LightningModule object, which led me to add self.trainer = Trainer(… to the subclass of LightningModule as you have in this codebase. But this leads to the strange syntax of self.trainer.fit(self, … . seems like that has worked out OK for you? I haven’t been able to figure out if that’s what other PL users are doing or if there are issues with it.
*Thread Reply:* I think being clear in docs that this is the desired format is a reasonable trade for being able to have a single config file that goes in and the user gets to call
self.create_trainer()
or some kind of wrapper and not need to specify the many many arguments.
*Thread Reply:* Of all the complaints we get, that has never been one of them.
*Thread Reply:* This paper is not related to MegaDetector, but it's the most comprehensive assessment I've seen of the benefits of AI to a camera trap project (it evaluates the use of the eVorta system wrt time, money, and carbon):
Smith J, Wycherley A, Mulvaney J, Lennane N, Reynolds E, Monks CA, Evans T, Mooney T, Fancourt B. Man versus machine: cost and carbon emission savings of 4G-connected Artificial Intelligence technology for classifying species in camera trap images. Preprint. 2024.
A couple of papers related to MegaDetector have estimated results as a speedup fraction to processing time... YMMV, they are all asking very different questions:
Fennell M, Beirne C, Burton AC. Use of object detection in camera trap image identification: Assessing a method to rapidly and accurately classify human and animal detections for research and application in recreation ecology. Global Ecology and Conservation. 2022 Jun 1;35:e02104.
Henrich M, Burgueño M, Hoyer J, Haucke T, Steinhage V, Kühl HS, Heurich M. A semi-automated camera trap distance sampling approach for population density estimation. Remote Sensing in Ecology and Conservation. 2023.
Mitterwallner V, Peters A, Edelhoff H, Mathes G, Nguyen H, Peters W, Heurich M, Steinbauer MJ. Automated visitor and wildlife monitoring with camera traps and machine learning. Remote Sensing in Ecology and Conservation. 2023.
[The last one doesn't explicitly talk about time, but because it's a rare scenario where full automation is plausible (because it doesn't involve species classification), time is implicit.]
But the way I usually look at this is to set aside all the papers that are about the use of AI for camera traps... any paper about AI for conservation (including papers written by me, or you, or most folks on this Slack) is biased toward a positive result, even if only through the review process. After all, we're here because we're excited about the potential of AI in this space!
Instead the case I would make is just "here's a bunch of people using AI who have no reason to if it didn't save them time". For that argument, I can only give you the MegaDetector-centric view, but FWIW, this is the reason I keep a list of MegaDetector users who agree to be included (or publish their use of MegaDetector):
https://github.com/agentmorris/MegaDetector/blob/main/README.md#who-is-using-megadetector
I want a new potential user to be able to look at that list, ping someone they at least sort of know, and ask about their experiences. Almost everyone on that list just had a job to get done, and had no inherent motivation to use AI.
Along the same lines, on the list I maintain re: papers about AI and camera traps:
https://agentmorris.github.io/camera-trap-ml-survey/#papers-with-summaries
...I have a tag called "ecology paper". That's not related to the journal it was published in, rather that tag is meant to capture "this person just had a job to do related to ecology, and wasn't inherently interested in AI, they just used it as a tool to do their work". (I realize I have not made it very easy to search for that tag... note to self.) Those are IMO better arguments in favor of AI than quantitative claims made in a paper about the use of AI: if those authors didn't find that AI was useful, they just wouldn't have used it.
*Thread Reply:* Avoiding the term “AI” and using something more specific like “automated image processing” could also help. “AI” comes with a lot of baggage these days 😅
👋 Hello everyone!
I'm Romain, a scientific researcher in the Behavioural Ecology Group at the University of Copenhagen, led by Elodie Briefer (https://www.behavioural-ecology-group.com/). My passion lies in leveraging machine learning to decode animals' emotions, and I'm thrilled to meet you and join this Slack channel! 😊
Beyond my research, I also specialize in web design, aiming to enhance online presence and professionalism. Recently, I had the pleasure of designing the AI & Bioacoustics 101 online workshop website, and I could not be happier to introduce it to you.
Understanding and protecting wildlife has always been a challenge. Traditional methods of monitoring animal behavior and communication can be time-consuming and invasive. This is where AI comes in—revolutionizing how we decode and interpret animal sounds, leading to significant advancements in conservation and welfare.
But how can we enhance animal welfare and conservation using artificial intelligence?
Join us at the Bioacoustics & AI 101 workshop on September 25th - 26th, where we’ll explore how AI can transform bioacoustics, making it more efficient and accessible. Whether you're a researcher, biologist, or simply curious about the intersection of technology and nature, this workshop offers invaluable insights.
Workshop Highlights: • Demystifying AI: Break down complex AI concepts into simple, accessible language. • Hands-on learning: Train your own deep learning network to classify sounds—no coding required! • Cutting-edge research: Explore the latest advancements in AI-powered bioacoustics. Featured Speakers: • Morten Goodwin: Professor of AI, University of Agder • Marie Roch: Head of the MAR Lab, San Diego State University • Anna Zamansky: Head of the Tech 4 Animals Lab, University of Haifa • Yossi Yovel: Head of the Bat Lab, Tel Aviv University • Léo Papet: Co-founder of Biophonia • Colleen Reichmuth: Head of the Pinniped Lab, University of California Santa Cruz Key Dates: • Abstract submission deadline: July 19th, 2024 • Acceptance notification: August 16th, 2024 • Registration closes: September 13th, 2024 ➡️ Don't miss this exciting opportunity to learn and connect with experts in the field. Register now for free and embark on an insightful journey into the future of AI in bioacoustics: https://aibioacoustics101.com/
Looking forward to virtually seeing you there! 🙂
Best,
Romain
*Thread Reply:* Being in Seattle after 7pm is getting close to my bedtime, but I'm so excited to see all of the AI4C folks that I'll make an exception to my "90-year old farmer" schedule.
*Thread Reply:* Fantastic, thank you for organising! 🙂
*Thread Reply:* Would have loved to join you and meet the community but paris-seattle makes for a long commute !
*Thread Reply:* Hi @Sara Beery as someone working on animal communication and based in Seattle, this sounds great! I’d love to join, but can I just show up or would have to register?
*Thread Reply:* I’m sorry I can’t be there (will be in Zurich)! But this sounds awesome!
*Thread Reply:* @Sara Beery Great, looking forward to seeing everyone!
*Thread Reply:* Amazing, for organizing this! I’ll plan to join + looking forward to meeting you all 🙂
*Thread Reply:* Sounds like fun. I'll pop across the ferry.
Hi everyone,
The third edition of the Computational Entomology Webinar will be running on June 20th at 1300 UTC. The overall theme of this edition of the webinar is processing liquid-preserved invertebrate specimens. Please see the flyer attached to this message for the speaker list, and you can find out more about the webinar (including registration) on our WILDLABS event page (https://tinyurl.com/CE-WebinarIII)
*Thread Reply:* As a note, we’re currently looking for participants with high ML expertise (e.g., have taken a course on ML and/or have experience working with a ML system, often use and study ML) and high birding expertise (e.g., have taken a course on birding and/or have experience in birding, often conduct bird-watching and study birding)
Hello everyone, I didn’t see an announcement for this on this slack (sorry if I missed it), so I take the liberty to advertise without the blessing of the organisers 😄 (@Mohamed Elhoseiny, @Sara Beery, etc): at ECCV 2024 in Milan there will be a workshop about Computer Vision for Ecology! I guess many people in this slack will be excited (I am!). Paper deadlines in July / August: https://cv4e.netlify.app
*Thread Reply:* Will there be a page limit for submissions? I did not find any information on the website. Maybe I missed something. Thanks!
Many of you may have already seen this, but the Bezos Earth Fund's $100M AI for Climate and Nature grand challenge is now accepting submissions, and biodiversity conservation is a focus area in the first round. Eligibility is limited to academic institutions and U.S. 501c3 nonprofits, but I imagine many of you have ideas and could partner with an organization to submit.
Feel free to shoot me a note if you have questions, or join the webinar on June 20th.
Come look at birds with us next Thursday morning before CVPR!!
https://x.com/sarameghanbeery/status/1802269672575783202
*Thread Reply:* For those who are driving, I suggest parking here: https://maps.app.goo.gl/WTjD63LQeL4RMfL29
*Thread Reply:* Make all of us who didn't make it this morning jealous... what awesome birds did you see?
*Thread Reply:* On Thursday!! No birds this morning, alas
*Thread Reply:* Ah, my bad. I mentally collapsed all of CVPR into "Monday" and "the rest of CVPR". I'll ask again for a bird report on Thursday.
*Thread Reply:* For those joining tomorrow, if you're coming over from the conference center with us we will meet at 7 in front of Arch (7th and Pike) and then take the light rail. Otherwise we will collectively meet at the UW Boathouse at 7:30!
*Thread Reply:* Will you leave at 7 sharp? Or is there a little time to pick up some breakfast to go?
Introducing the first Computer Vision for Ecology (CV4E) Workshop at #ECCV2024 in Milan 🌳 If your work combines computer vision and ecology, submit a paper and join us! The workshop will cover applications across diverse ecological systems, featuring exciting speakers, discussions, and challenges (TBA). Thank you @Devis Tuia for promoting us earlier this month 🙂
Deadlines: July 15 (Proceedings) / Aug 15 (Non-Archival) More details: https://cv4e.netlify.app/submit/
Also please follow our twitter page for future updates! https://x.com/CV4E_ECCV/status/1802921207286862026
*Thread Reply:* To everyone interested in submitting, we have just updated the website with more submission details such as the paper formatting requirements! Please see https://cv4e.netlify.app/submit/
Also, the paper deadlines are still July 15th and Aug 15th for proceedings and non-archival respectively 🙂 Looking forward to everyone's works
Hi guys, maybe some of you have probably seen this before but I came across this really interesting paper which demonstrates that African elephants address each other with individually specific name-like calls. Here’s the link if anyone is curious: https://fermatslibrary.com/s/african-elephants-address-one-another-with-individually-specific-name-like-calls#email-newsletter
*Thread Reply:* You have so many cool talks, I might have to jump in and out as I have some colleagues with overlapping talks but looking forward to catch as much as I can of your symposium!
*Thread Reply:* Hi! Check out fine-grained classification problems https://paperswithcode.com/task/fine-grained-image-classification CUB and NABirds datasets are a place to begin your search
*Thread Reply:* that's an amazing website ty so much !
*Thread Reply:* Hi, we've been building a large semantic segmentation dataset (of coral reefs) with the help of multiple experts. We're using CVAT (https://github.com/cvat-ai/cvat) which we're hosting on one of our own machines. CVAT works great, and allows configuring in a way to have projects with users and assign them sets of images etc. We also use CVAT's 'Segment Anything' support (runs on the GPU on our machine), which helps a lot.
*Thread Reply:* We had used labelme for easily annotating degradation indicators in a degraded tropical forest. Super easy to use tool. Straightforward. Allowed for offline usage (one of our annotators had limited access to internet). But doesnt have a lot of the modern features that come with cloud-based or self hosted softwares.
https://github.com/labelmeai/labelme
For working with multiple annotators, we didnt have them labelling the same images. Also, every annotator was considered equal to the other in terms of their ability. However these two docs might be interesting reads:
https://docs.cleanlab.ai/stable/tutorials/multiannotator.html https://docs.aws.amazon.com/sagemaker/latest/dg/sms-annotation-consolidation.html
Hi all, During CVPR, Sara swung by AI2 and gave a lecture on a new benchmark dataset for natural world imagery called INQUIRE.
Abstract: Natural world images collected by communities of enthusiast volunteers provide a vast and largely uncurated source of data. For instance, iNaturalist has over 180 million images tagged with species labels, already contributing immensely to research such as biodiversity monitoring and having been cited in over 4,000 scientific papers. Yet, these images are also known to contain a wealth of ""secondary data"" captured unintentionally or otherwise included in images and not properly reflected in image labels. Although this data contains crucial insights into interactions, animal social behavior, morphology, habitat, co-occurrence, and many more questions, the costly, time-consuming, or expert-dependent analysis needed to extract such information prevents breakthroughs. Advances in deep learning methods for language and computer vision have the potential to enable the efficient and automated processing techniques needed to unlock the ""hidden treasure"" in such datasets– being able to directly search large image collections for these concepts would enable richer analyses that span beyond species identification.
We posted the talk here: https://www.youtube.com/watch?v=TnOrlfRgmv4. A heartfelt thank you to Sara and her team for their visit. It was truly a pleasure to finally meet many of you in person after a year of virtual interactions.
ECCV 2024 Workshop Extended Deadline + Updated Requirements! We are extending the archival deadline to July 22nd. We are also now accepting Full Length Papers (14 pages) for the archival track, in addition to Short-form Papers (7 pages).
Please see https://cv4e.netlify.app/submit/ for final details
Hi All, I spent my 4-week sabbatical from Arm helping researchers in Ecuador accelerate their camera trap analysis using AI. We started with with simple to use application like EcoAssist and are working toward an Ampere-based Biodiversity Server. Read more here: https://www.linkedin.com/pulse/bringing-bearid-ecuador-sabbatical-story-ed-miller-tkh5c/?trackingId=t7Ylo0FcRt6YGRlOJYNQDg%3D%3D
*Thread Reply:* @Ed Miller i've worked with a number of lodges in Ecuador, I forwarded your post to them, you may hear from Maquipucuna/Santa Lucia lodge where there are long standing camera trap programs and plenty of andean bears.
*Thread Reply:* @Ed Miller that is such an amazing project. You are truly living my dream. Do you have any advice on how to make the kind of connections that led to your involvement in the BearID project? I transitioned from a career in animal behavior research to focus on software development with ML and am looking for more project opportunities in animal science.
*Thread Reply:* @Kara Watts I made my initial connections on https://www.wildlabs.net/. You can dig around the various forums there to look for ideas and collaborators. You can also start working on something that interests you and reach out there for help!
*Thread Reply:* @Ed Miller Thanks for the advice!
*Thread Reply:* @Dan Morris has a pretty nice list somewhere I'm sure 🙂
*Thread Reply:* As always, I live for making lists, or in this case, even better, a list of links into another list...
Within the list I maintain re: tools related to AI and camera traps:
https://agentmorris.github.io/camera-trap-ml-survey/
...there are a few relevant sections. The first is the list of image review systems that use AI in some way, which in 2024 is almost all of them:
https://agentmorris.github.io/camera-trap-ml-survey/#camera-trap-systems-using-ml
If a tool is open-source, there should be a link to the source in the description of that tool.
There is a second section that lists no-AI-at-all tools, but as I look at this section now, I think "no AI at all" is almost synonymous with "deprecated". I.e., even the tools that have been updated in the last few years are no longer in use as far as I know, at least I haven't heard of anyone using them recently, with the possible exception of Camera Base (which hasn't been updated in a while AFAIK, but I still hear about folks using it):
Last but not least, there's a section for tools that aren't specific to camera traps that people sometimes use for reviewing camera trap images; among these, Exif Pro is the most widely used AFAIK and the only one that's open-source:
I have no real data to back this up, but my sense is that among desktop tools, Timelapse has the largest "market share" on Windows, and Exif Pro has the largest "market share" on MacOS, although Windows is still far more widely used than MacOS for reviewing camera trap images.
*Thread Reply:* See??? @Dan Morris always coming in clutch with the lists 🤩
*Thread Reply:* thanks so much you both, this was so helpful!!
Hi! I'm not sure if this has been shared here already. Here is a nice dataset of aerial imagery combined with height and density data for forests https://arxiv.org/abs/2407.09392v1?utm_source=tldrai
Hi all, I've published a blog post on rapidly labeling camera trap data using ChatGPT for species identification and a simple object detection model to get the bounding boxes. With this setup, I could run through a set of camera trap recordings and automatically obtain 520 labeled images with detected animals, with only some minutes of manual work by controlling the output, and batch relabel samples of the same species but got different spellings, e.g. "Alaska moose" and "Alaskan moose".
While presented in Edge Impulse, this approach can be replicated outside the platform using Python scripts for example. I’d love for you to give it a read and share your thoughts on its potential usefulness in real-world applications.
video: https://www.youtube.com/watch?v=Ek1MmZIvtE&t=1s|https://www.youtube.com/watch?v=Ek1MmZIvtE&t=1s and blog post: https://www.edgeimpulse.com/blog/adaptive-camera-trap-gpt-4o/
*Thread Reply:* i see this message on the <#CMAFLU078|animal_re-id> channel, fwiw. doesnt seem to say "delete" but older content is unavailable.
*Thread Reply:* it links to this page which does seem to imply there is a paid plan which solves the problem.
*Thread Reply:* ( or we could move to something open source like matrix. 🙂 )
*Thread Reply:* Actually on second review of my Slack deletion notice emails it might be only for Slack channels that haven't had any activity recently. However, it's still a bit unideal that we can't tap into all the old content of this channel.
Do people encounter the issue where they run into messages they can't see often? What are the usecases where this happens?
*Thread Reply:* @Kalindi Fonda Free slack teams already have a limit where you can't access messages over 90 days old. However Slack has been hanging on to those messages behind the scenes, probably to incentivize you paying to unlock them and access your full history. This new policy is that after August 26, they will delete the archived messages that are older than a year, so even if you switched to a paid plan, you couldn't access your older team history. There was a discussion of changing platforms for this slack, but ... it took place more than 90 days ago..! ;^) (Hope this explanation was not too mansplainy — their change from keeping 10,000 messages to only 90 days really impacted a lot of science communities, so I've been following the topic.) https://www.yahoo.com/tech/slack-delete-chats-files-free-105338864.html
*Thread Reply:* Thanks @Steve Haddock, yes I've been part of quite a few communities when the previous switch (to 90 messages) happened, so I remember the impact 😢
*Thread Reply:* So are all slack communities affected? (I got a couple emails, I guess it's for slack channels where I am the owner/admin). There is the option to export the conversations, is this something the admins would be interested?
*Thread Reply:* for those of us unfamiliar with the “moderator” role, is there a description somewhere?
🚨 New LILA BC dataset alert!
Thanks to the guys at Desert Lion Conservation, we got a new dataset consisting of 65k images and 200 videos of Namibian fauna. Labels are provided for 46 categories, primarily at the species level. The labels are mapped into the shared taxonomy.
@Dan Morris thanks for your help!
More info: https://lila.science/datasets/desert-lion-conservation-camera-traps/ MD results: https://lila.science/megadetector-results-for-camera-trap-datasets/
Cross-posting here, ESA anyone? https://aiforconservation.slack.com/archives/CM1JPL18R/p1721939084318559
}
*Thread Reply:* is there a conservation tech meetup planned?
*Thread Reply:* There should be!! But I've been slammed and haven't planned anything. How about meeting for beers Monday evening at 7ish somewhere near the convention center?
*Thread Reply:* Ok, lets do it. ESA Conservation Tech Meetup at 7pm Monday August 5th, at Altar Society Brewing
*Thread Reply:* https://x.com/sarameghanbeery/status/1817685293249740829
*Thread Reply:* https://wildlabs.net/event/ai-conservation-meetup-esa-2024
*Thread Reply:* Hope to see many of you tomorrow!
Reposting details:
ESA Conservation Tech Meetup at 7pm Monday August 5th, at Altar Society Brewing
https://maps.app.goo.gl/QwBKQPofRKQJbGLj7
*Thread Reply:* It was great to see everyone last night!!!
*Thread Reply:* Unfortunately, due to some nightmare travel disruptions I wasn't able to get here in time for last nights social, if anyone wants to connect over a coffee or beer over the next few days, feel free to message me!
*Thread Reply:* Do you mind if we cross-post this elsewhere?
*Thread Reply:* @Sara Beery Do we have a list of PIs, as well as their research groups and labs and institutions, who will be joining the session? I hope to do some research into the work that those PIs and groups do before the meeting, so that I can make the most of our time together during the session!
*Thread Reply:* Good idea if we do this again! I will share the videos on youtube after, so that will be a more permanent resource, but because there has been some flux about which faculty will join live vs record videos for later I won't be able to share a list ahead of time
*Thread Reply:* got it, thanks Prof Beery!
*Thread Reply:* Oh no I missed this! Has it been recorded?
*Thread Reply:* Yes, recordings should all be up hopefully this week
The Joint Statistic Meeting is here in Portland and @Toryn Schafer and I just had a nice conversation on how to get more integration between the statistics community and AI4Ecology. @Ben Augustine and others jump in. Just starting a thread here in case others want to be involved (@Eric Orenstein). Areas of intersection include 1) multi-observer processes that use both human and automated observers, 2) finding creative solutions to move toward hierarchical models that capture uncertainty, 3) Going beyond cross-validation and boot strapping as a way of assessing prediction confidence intervals. 4) Merging automated detections with existing time-series of human observations. If there are others in the community interested in solving these kinds of challenges, see this funding opportunity. https://new.nsf.gov/funding/opportunities/computational-data-enabled-science-engineering. I think many of us would be happy to support proposals that bring greater quantitative rigor into using the outputs of these models. I'd like to start a quick literature thread here to bring community knowledge together. https://www.biorxiv.org/content/10.1101/2023.02.20.529272v1.abstract, https://www.mdpi.com/2504-446X/8/2/54, https://zslpublications.onlinelibrary.wiley.com/doi/full/10.1002/rse2.356, if others can drop papers in here they have authored, or know of.
*Thread Reply:* It may be outdated, but I believe @Ben Augustine had done a lit search related to thrust 2 for a Powell Center working group 4 years ago
*Thread Reply:* Our group is very interested in this area, would be great to chat about it - particularly area 1 and adding variations of marked abundance models with uncertain marks
*Thread Reply:* My group is interested as well, particularly @Timm Haucke and @Justin Kay
*Thread Reply:* Wild Me is definitely interested in this. I will bring cookies.
*Thread Reply:* This is great. I’d be very interested in chatting more about uncertainty, data fusion, and hierarchical models
*Thread Reply:* Hi folks, we’re also interested in this topic at UMass. We have a couple related papers (1, 2), and would be glad to chat with folks. @Gustavo Perez @Subhransu Maji @gvanhorn
*Thread Reply:* @Dan Sheldon @Gustavo Perez, we have airborne modeling work for BOEM (https://www.fisheries.noaa.gov/inport/item/67243), i'd be interested in trying a lightweight versus of this for their biodiversity surveys. I will DM you once I read closely.
*Thread Reply:* Also interested! I spend a lot of time thinking about detection probability on the acoustic side of things, esp. with respect to human & AI integration. Would love to be kept in the loop (although we’re in Canada, so not eligible for NSF)
*Thread Reply:* My group is interested. We've put out a few of papers that could potentially fit: • BirdSAT: Cross-View Contrastive Masked Autoencoders for Bird Species Classification and Mapping (WACV 2024): https://arxiv.org/pdf/2310.19168 • LD-SDM: Language-Driven Hierarchical Species Distribution Modeling (still a preprint): https://arxiv.org/pdf/2312.08334 • PSM: Learning Probabilistic Embeddings for Multi-scale Zero-shot Soundscape Mapping (will be published at ACM MM 2024): https://openreview.net/pdf?id=qnW0LQXY5L
*Thread Reply:* Would also be interested in being involved! Very relevant to my current research.
*Thread Reply:* I am interested in submitting an invited session proposal for the next JSM focusing on the thrusts 2 and 3. The JSM 2025 theme is "Statistics, Data Science, and AI Encriching Society". We could do a paper session or panel. Here is more information: https://ww2.amstat.org/meetings/jsm/2025/invitedsessions.cfm#invited
*Thread Reply:* Would also like to be kept in the loop!
*Thread Reply:* We introduced it in the Recognition in Terra Incognita paper, in the appendix, so you could cite that I guess?
*Thread Reply:* But @Dan Morris looks like we should update the links on LILA for the format
*Thread Reply:* I have a specific section that I link to for this:
Sara correctly points out that a few format links on LILA have become stale. Good catch, I will fix that in a few days, and also probably replace with a shortlink.
*Thread Reply:* I've fixed this error in all the places it occurred on LILA, and you can use the following shortlink now to refer to the CCT format specification:
https://lila.science/coco-camera-traps
Thanks again @Dan Stowell for catching this.
*Thread Reply:* Thanks! btw, Slack recently increased the 90 days to 365.
*Thread Reply:* Interesting, I’m still seeing 90 days, but let’s hope this change is indeed rolled out!
CALL FOR EXPERTS AI Tools for Addressing Conservation and Biodiversity
Dear Colleagues,
The
We will begin reviewing nominations by August 7 COB. Please submit nominations via the
Toward a Resilient California AI Tools for Addressing Conservation and Biodiversity
Biodiversity is declining globally at unprecedented rates due to a variety of cascading and compounding anthropogenic impacts including climate change, habitat destruction, pollution, and overexploitation. To address this crisis, governments around the world are committing to the 30 by 30 initiative with the goal of conserving 30% of the world’s lands and waters by 2030 to protect and restore biodiversity. High quality biodiversity data on the distribution of animals and ecosystems is vital to successfully meeting these goals but is often costly or time consuming to collect. Advances in the fields of artificial intelligence and machine learning are a promising avenue for supporting researchers and practitioners who are building these datasets.
This Expert Briefing will explore examples of novel research projects that are utilizing artificial intelligence tools within the conservation and biodiversity space to augment ongoing efforts to collect robust datasets. The discussion will explore: lessons learned and challenges of incorporating AI into conservation work, opportunities for using these tools to inform management decisions, and pathways for leveraging civic science to support the development of this valuable conservation work.
CCST seeks nominations of individuals (including self-nominations) with relevant expertise who can give background on these topics and potentially serve as a briefing panelist or moderator.
The session in ON! Please join us
}
*Thread Reply:* Fantastic event!! thanks to Sara and all organizers!
*Thread Reply:* Where will I be able to find the recording?
AI and Ecology Research Lab introductions now LIVE on YouTube!!!
Check it out : https://youtube.com/@AIforConservation/playlists…
And if you're a PI who works in AI+Ecology and want to be included, send me a 3-min intro video and we will add it!!
*Thread Reply:* That was great, thanks to all presenters and to @Sara Beery for organising! Caught everything on the Youtube channel just now.
Hey guys! I just wanted to let you know that we have published the final program for the 4th International Workshop on Camera Traps, AI, and Ecology! 📅 https://camtrap2024.fh-ooe.at/program/
Almost at the same time we also received the hundredth registration🎉 However, registration is still possible so, join us from September 5-6 at the Hagenberg Campus - FH Upper Austria or online for a deep dive into cutting-edge tech and ecological research. 🌍💡
Our hybrid agenda includes: • Four incredible keynotes by top experts such as Robin Sandfort (capreolus e.U.), Cliodhna Quigley (Universität Wien | University of Vienna), Stefano Mintchev (ETH Zürich), Claudia Probst and Georg Roman Schneider (Fachhochschule Oberösterreich) • Three tutorial sessions by Danielle McKenney (SWAROVSKI OPTIK), Andreas Leitner and the Smartmulticopter team, as well as Piotr Tynecki (TRAPPER) • Four paper sessions including ten incredible submissions in the realm of camera trapping, insect monitoring, drone based wildlife monitoring as well as analysis of forests • Social Event and dinner as part of this year's Ars Electronica Festival (only available for registered on site participants) This is a fantastic opportunity for collaboration and innovation in wildlife monitoring, ecology research, and more. 🦌🐝🤖 Let’s shape the future of conservation together!
Join us and be part of the conversation!
*Thread Reply:* This is happening now (online too) but in case anyone is here would love to say hi 👋 🌱
*Thread Reply:* I didn't manage to call in to this event... folks who were there physically or virtually, share something neat you learned/heard/saw/said/met/detected/classified?
*Thread Reply:* Oh no sad to hear that @Dan Morris. I hope due to time issues and not because of technical problems 🥲 Since I had the honor to host the event, I don‘t think I should say my opinion too loud (self-praise and so on 😂 ), but I really think we had some pretty awesome presentations within the workshop 🙂 FYI: I will upload all the recorded presentations the next days, so if somebody missed the workshop, you can rewatch at least the presentations on our homepage http://camtrap2024.fh-ooe.at (unfortunately withe one exception due to a missing agreement on recordings from the company).
*Thread Reply:* Clarifying: yes, just time zones and my own inertia, not technical issues. I'll keep an eye out for the recordings, thanks!
*Thread Reply:* Still sad but I am calmed down that it was not because of technical issues 😄
Little follow up, I have quickly used the time to upload some impressions of the workshop as well as the presentations and papers: https://camtrap2024.fh-ooe.at/gallery/ https://camtrap2024.fh-ooe.at/program/
The actual proceedings are currently in the making, but this will still take some time 🙂
*Thread Reply:* I always say the reason I am into tech is the scalability and the leverage it provides, and I usually think about it from the point of view of how by creating tools they become accessible at scale...
There was a point made by @Robin Sandfort that really resonated, he was talking about how AudioMoth being cheap compared with highend devices, means that one can put many more out instead of one single highend one. And sure there is a bit of a loss in the quality of the recording, but one might have never gotten a recording in the first place with only one location being monitored.
Then we continued the conversation and Robin shared a story about how one area where he had installed some AudioMoths has been flooded, and that some devices survived, so he now has the "sound of a flood": the sounds of water rushing in, the frogs being closer and closer, and then the gluglu and then the whole thing in reverse.
The chance that one high end device would break in a flood is a certain amount, but the chance that all the AudioMoths would break in that flood is much lower.
So the point that was elusive before but has now been added to my "I <3 tech because scalability" is redundancy , And probably there are many ways to think about it, also in terms of knowledge, or types of projects...
*Thread Reply:* Plus the organisation was incredible. The whole conference was gentle and caring, the talks were interesting, and covered various topics.
Then at the end of the first day we had a bus for us that took us to town, to an interactive art exhibition (which provided for external stimulus to further conversations), and then a dinner under some trees, where there were lots of chats and smiles.
I'll take a look at my notes I know I wrote down a few quotes of expressions I liked, and maybe even some more on the content. 🌟
*Thread Reply:* As a personal aside, I've been trying to rope my vet technician friend into a more active role in relationship to nature/conservation. She had studied forestry, and is now working at as a vet technician, but I know she has "the call of the forest". I was sending her pictures and insights, and I think something is starting to shift within her 🌱
*Thread Reply:* Really glad to hear that @Kalindi Fonda! It was a pleasure having you at the workshop and meeting you 🙂
Does anyone know what the best way is to individually recognise Orangutans from images? My gut tells me to zoom in on the face, but I can't really find any literature about that.
*Thread Reply:* Light Internet search suggests faces (primarily) with some challenges around certain age classes. If there is a dataset of individuals, you might try our multispecies model, which includes faces for some species:
https://huggingface.co/conservationxlabs/miewid-msv2
*Thread Reply:* I think @Otto Brookes has been working on face recognition for gorillas. Maybe a good person to contact!
*Thread Reply:* I got some "good" result using MegaDescriptor with bear faces, could be nice to compare Miewid and MD.
*Thread Reply:* Is the bear face dataset open?
*Thread Reply:* https://huggingface.co/conservationxlabs/miewid-msv2
*Thread Reply:* Nope, it was from the BearID proyect, I have access via the AI for Bears Fruitpunch Challenge, @Ed Miller could say you more about the permision of the dataset.
*Thread Reply:* If there's only a single Orangutan in each image (like the one shown), I would just train a model on the whole image! If you have multiple apes in each image then you'll likely need to localise them (either their face or full body). Happy to discuss further if it's helpful!
*Thread Reply:* Thanks for the info everybody! I'll play around with the models and if needed will get in touch with you @Otto Brookes 😁
*Thread Reply:* Hi Peter, together with Dan Schofield from Oxford university a group of orangutan researchers is working on this with some large datasets from wild orangutans. It follows the work Dan did on chimpanzees.
*Thread Reply:* Just get in touch and I can give you more details.
*Thread Reply:* @Serge Wich , that sounds great! I’ll discuss this with the other project members and reach out to you via email.
*Thread Reply:* we have a coupl eof the editors in this Slack, maybe DM them with your inquiry 😉
*Thread Reply:* Maybe @Majid Mirmehdi or @Otto Brookes can tell you more 🙂
*Thread Reply:* Dear @Rita Pucci,
I sincerely apologise for the delays with your paper. This is highly unusual and our turnaround times are normally well below 3 months.
The Editor handling your article has received 35 “decline”s from nominated reviewers which in itself is highly unusual. There is one review available and we are trying to push another referee whose review we are waiting for. I hope we can be in touch with a decision soon.
Best wishes, Majid
*Thread Reply:* Ok thank you for your interest
*Thread Reply:* This is a very camera-trap-specific answer, but...
If there are folks in the audience who aren't Python-savvy and may be intimidated by Colab, consider demo'ing EcoAssist, or at least highlighting that you don't need to deal with any Python to use AI.
But I'm not discouraging you from also showing stuff in Colab. FWIW if you haven't come across it, we have a pretty straightforward MegaDetector Colab notebook here.
*Thread Reply:* :gratitudethankyou: that's a great suggestion! I agree that diving right into a colab notebook might turn some folks away; so I'll start by demo'ing EcoAssist and timelapse.
And megadetector_colab.ipynb is indeed the notebook I was planning to demo 🙂
I created a barebones classifier demo using some scripts in your megadetector repo but maybe there's a megadetector_classifier.ipynb demo somewhere that I'm missing?
*Thread Reply:* <unsolicited advice that isn't what you asked for>
Training a custom classifier is IMO a little advanced for an introductory session, i.e. it may overstate the degree to which you need to train a custom classifier (which is not typically worth it... IMO in 95% of cases you will save more time overall by using either no classifier or a classifier that's "close enough" than by doing all the work required to train a custom classifier). So instead, consider using the (now very impressive!) model zoo that's built into EcoAssist to demonstrate the growing variety of classifiers that are already out there, and the way you might use the results from a species classifier in Timelapse.
</unsolicited advice>
To your actual question, there isn't a notebook for training a classifier within the MegaDetector repo. To each their own, but for my two cents, classifier training in a notebook is a tough putt, it's such a fundamentally asynchronous and infrastructure-dependent experience. So if you want to demonstrate workflows for training classifiers, consider demo'ing one of:
MEWC (exactly what you want, but not easy to demo in a notebook): https://github.com/zaandahl/mewc
Zamba Cloud (for video rather than stills, but still a really slick workflow for no-code training of custom classifiers): https://www.zambacloud.com/
All that said, if you do end up putting together a notebook to demonstrate classifier training, please share here!
There is also this notebook (a couple years old) that fine-tunes MegaDetector to add species categories; the notebook is nice, although this is no longer the approach I would recommend:
https://www.kaggle.com/code/evmans/train-megadetector-tutorial
Also, unrelated to training, if it's a pretty broad session, consider reminding folks that they don't have to do everything on their laptops in 2024, i.e. for folks who might be up for a cloud-based solution for AI, image review, and data management, consider demo'ing, e.g., Wildlife Insights.
*Thread Reply:* Really appreciate all these suggestions! I wasn't planning to make a colab notebook for training a classifier, just inference with a pretrained model (e.g. your megaclassifier-efficientnet-b3 model file). I guess that's pretty straightforward.
*Thread Reply:* Got it, all clear, ignore my unsolicited advice then. 🙂
There is not exactly a notebook for running MegaClassifier, but... there is a notebook I use for doing All The Things involved in a complete detection+classification batch, and "All The Things" includes running MegaClassifier:
https://github.com/agentmorris/MegaDetector/blob/main/notebooks/manage_local_batch.ipynb
I actually use the .py version, but it's the same as the .ipynb version (the latter is auto-generated from the former):
https://github.com/agentmorris/MegaDetector/blob/main/notebooks/manage_local_batch.py
This notebook doesn't run MegaClassifier directly from Python; rather, it writes a shell script (.bat or .sh, depending on the OS) that does the MegaClassifier stuff. See the cell called "Run MegaClassifier (actually, write out a script that runs MegaClassifier)".
*Thread Reply:* @Atul Ingle feel free to include TrapperAI species model in your edu program after animal recognition. It is easy to load and use in notebook/colab, if the list of supported spieces is covering your context.
*Thread Reply:* @Atul Ingle If you were planning on using https://github.com/zaandahl/mewc to train classifiers, note that there is an option of loading the resulting models into EcoAssist for inference.
https://github.com/PetervanLunteren/EcoAssist/blob/main/markdown/MEWC_integration.md
<a href="https://github.com/PetervanLunteren/EcoAssist">PetervanLunteren/EcoAssist</a>
Wildlife.ai has a dream job opportunity🤩🤩🤩 Do you want to be the next Product & Community Manager of a thriving non-profit? Apply now! #jobs #nz #community #grassrootsconservation
*Thread Reply:* We play in this area in attached. Its far from the kinda of hierarchical model that we want, but starts to gain some insight. This is kind of thing we were discussing @Toryn Schafer , @Ben Augustine, trying to couple non-calibrated scores with downstream confidence intervals. @Eric Orenstein has a paper on plankton that orbits this space as well. TLDR, nothing that i'm aware does precisely what you are looking for. Also summoning both @Casey Youngflesh and @Heather Lynch, since both are close colleagues and are in this channel, and you pointed at their paper.
*Thread Reply:* fantastic, thank you!
*Thread Reply:* I've used multiple imputation in the past to propagate a first stage prediction uncertainty to a second stage inferential model: https://doi.org/10.1007/s13253-020-00399-y
*Thread Reply:* This paper is not about a detection model, but otherwise it gets at what you're asking about (the relationship between ML accuracy and downstream metrics):
Whytock RC, Świeżewski J, Zwerts JA, Bara-Słupski T, Koumba Pambo AF, Rogala M, Bahaa-el-din L, Boekee K, Brittain S, Cardoso AW, Henschel P. Robust ecological analysis of camera trap data labelled by a machine learning model. Methods in Ecology and Evolution. 2021 Jun;12(6):1080-92.
*Thread Reply:* thanks, these are all really great references!
*Thread Reply:* I don't have any paper suggestions but here's my (extended) take. I'd probably add another level the traditional mark-recapture framework. You've got your observation model, your process model, and you could add the 'classification model'. I could see adding something to eq. 4, though I think the classification process is different from the observation process. Also, I think this matters for the creation of that vector z. Curious what folks think about the below, where t is time step:
y = ML classified observed state (given as data) m = latent true observed state (estimated) z = latent true occupancy state (estimated) p = probability you observe, given occupancy (estimated) psi = probability occupancy (estimated) h = probability that classification is correct (given as data) If yt = 0, ht = True positives / True positives + False negatives If yt = 1, ht = True negatives / True negatives + False positives
yt ~ Bern(mt * h_t) m_t ~ Bern(z_t * p) z_t ~ Bern(psi)
The one issue here is that you need that vector of z's (latent true occupancy over the the series) which you get from the entire time series of true observed states (filling 1s between first and last observation, assuming a period of closure). That's typically fed to the model before hand (as data). And you don't know the entire time series of true observed states (m's) until your model has run through the time series -- typically your data are your true observed states. Maybe an approach similar to that taken by @Toryn Schafer above could work there for the z vector?
*Thread Reply:* We did something slightly tangential. We used a bayesian characterisation of a detector, but we weren't so much interested in precision/recall but more in the location accuracy. Assume you have a true positive from a multibox detector, most papers only look into minimum jaccard overlap to count it as a true positive (0.5, 0.95, ...) or maybe a weighted distribution to compare models, but when you want to use the localization downstream, you need the localization error distribution of the detector. We did that in https://doi.org/10.1109/LRA.2018.2850224 - the result is with close approximation a normal distribution - when measured over the evaluation set across different scale ranges:
*Thread Reply:* the false positives fall into the tail edge of the distribution
*Thread Reply:* that way we can use the projected position with a bayesian tracker and have a good estimate of the "sensor noise" when the DNN is the sensor
*Thread Reply:* @Casey Youngflesh that is roughly the likelihood of a multiscale occupancy model where the availability component is recast.
Different context, but @Tessa Rhinehart did this with bioacoustic predictions [https://besjournals.onlinelibrary.wiley.com/doi/10.1111/2041-210X.13905] where continuous scores of reviewed files were modeled as a normal 2-part mixture of the logits.
*Thread Reply:* I smell a review piece here if any graduate students are lurking. @Casey Youngflesh my problem is thinking about those latent states when you have more than just presence/absence. Ignoring for a moment that confidence scores are non-normal and only look like probabilities (already a fatal flaw), and that not all classes are confused equally (also fatal), the multinomial probability isn't just whether the classification is correct (h in your psuedo code), because it also adds to another class. So for example in our tree work, if we want to count all the trees in a location and the classification model incorrect predicts it to be an oak tree, but its a maple, it both effects the undercount of oaks, which is covered in your model, but also the overcounts of the maples. You can easily image any mcmc process just endlessly trading off when trying to estimate that true latent state (m), no identifiability, especially as you get to more than 2 or 3 species.
*Thread Reply:* Important to distinguish sigmoid or soft max type heads, where that tradeoff is different
*Thread Reply:* thanks for all this, definitely a lot of good papers I’ll have to go read. This type of question has certainly haunted me before, and I would be interested in a review paper if one ever appeared @Ben Weinstein 🙂
*Thread Reply:* @Ben Weinstein Totally agreed about confidence vs. probability of correctness. But if you did have a proper probability of class in a vector [p sp1, p sp2, p sp3], that should account for the fact that species aren’t confused equally, right? Like, if the model knows they aren’t confused equally that should be baked into the probabilities (I see how it might not know though). I also agree that numerous classes are bound to screw things up. But I’m not sure I follow why you’d necessarily expect to have a non-identifiability problem.
Maybe I don’t understand the problem, but for trees in a given patch, you have a vector of probs for each tree. You draw a value for z (latent true species identity of a given tree i) from this vector of probabilities for tree i (let’s just assume these are actual probabilities, one for each species possibility). Maybe you do this 10k times. So for each tree i, you have 10k realizations of true species ID. Then at each of the 10k iterations, you sum the number of species of each tree. If you’re just interested in counts, you have 10k realizations of counts for each species. This is just being backed out of the (assumed actual) probs, not with a Bayes model.
If you wanted to model the composition of trees as a function of stuff, you could use a Bayes model with a Dirichlet, yeah? With each of the proportions having an error model (you would need a sum to zero constraint though)?
*Thread Reply:* Don't know if this is particularly helpful (most of this discussion is above my head), but the statistician in my group has used this in the past (conformal prediction for estimating confidence intervals): https://arxiv.org/pdf/2107.07511 At least in my understanding, if you get CIs on the model prediction (rather than probabilities), you should be able to propagate that uncertainty forward into the downstream analyses.
Also appears first author is in this slack channel.
*Thread Reply:* Just catching up with the discussion! For the model sketch from @Casey Youngflesh, I tried having a friend go down that route for a multi-stage model with classification errors. The model software wasn't happy with the specification so we ended up just defining each probability assuming independence of the classification and observation processes, we had: P(y = s| latent = s) = P(detected and classified as s| latent = s ) = P(detected | latent = s)P(classified as s)
*Thread Reply:* This is probably also prone to some of challenges due to the simplifying assumptions pointed out above
*Thread Reply:* Had you seen this list before? It has a bunch more as well https://bioacousticsdatasets.weebly.com/
*Thread Reply:* It would be good to specify what you mean by ‘dataset’ too - species-specific calls, ML training datasets, annotated or un-annotated, etc.? Like are you wanting Xeno-Canto, Macaulay library, etc. there too?
*Thread Reply:* Yes, I did see it! I included their datasets in my list. 😊
> Like are you wanting Xeno-Canto, Macaulay library, etc. there too? Good point, I should make it clearer in the repo. I'm not considering databases or repositories like Xeno-Canto whose content changes over time, I am considering ML-ready datasets. Both labeled and unlabeled data are fine. I'll update the README tomorrow!
*Thread Reply:* In that case, I would also recommend cross-checking your list with LILA BC too!
*Thread Reply:* there are a number of ML-ready annotated datasets there as well - https://lila.science/otherdatasets#bioacoustics
*Thread Reply:* Might also be a good idea to post about this list on WILDLABS too!
*Thread Reply:* > In that case, I would also recommend cross-checking your list with LILA BC too! Yup, already done that! I used their list to add some datasets 😊
*Thread Reply:* There are a bunch more datasets out there: https://zenodo.org/search?q=Klinck&f=resource_type%3Adataset&l=list&p=1&s=10&sort=bestmatch Scroll through the list...
*Thread Reply:* I will go through it, thanks!
*Thread Reply:* this is great, I’ve added it to the Bioacoustics resource page
here’s one more list by @Tessa Rhinehart to check https://docs.google.com/spreadsheets/d/1KrmCB0vvSK7V3znJfycO-eOMZJKP2F-Ih6neRYPz1Xc/edit?gid=0#gid=0
*Thread Reply:* Thank you very much! I have already gone through Tessa's list when I curated the datasets list. I think I will mention in the README all the resources I have gone through to compile my list. 🤔
*Thread Reply:* Just submitted a pull request for our coral reef dataset 🪸
*Thread Reply:* Thanks! 😍 By the way, if you found the procedure to add a dataset or anything else unclear, don't hesitate to tell me!
*Thread Reply:* Great work! Just a small comment, it is hard to read if you have a “dark-mode” ON in your browser.
*Thread Reply:* That's not hard, that's impossible 😂 I didn't think about that, thank you for raising the issue, I'll look at mitigating it!
*Thread Reply:* Mmmm, lists of datasets are like my favorite thing (just behind baby animals, guitars, and football). I added a link to your page from the list of acoustic datasets on LILA's "other datasets" page (the same one linked to earlier on this thread):
*Thread Reply:* I am also open to hiring a post-doc for this position in lieu of a graduate student.
Hi all, If a PhD student here with computer vision background is looking for a 3-month internship in a project related to CV and Robotics for Conservation, and is based in the EU (preferably) please DM me asap. I will be happy to recruit someone from here before posting the ad widely. For more info on the project in which the internship will be embedded, please check: https://www.aamirahmad.de/projects/wildcap/
*Thread Reply:* Thanks Dan, I've already advertised it above 😁 (https://aiforconservation.slack.com/archives/CLWGQ4BJ6/p1725894105104259)
}
*Thread Reply:* Great! (and sorry for double-posting) Thanks everyone for your comments on Celine's post
*Thread Reply:* Related event on bioacoustics data analysis and AI happening tomorrow! https://wildlabs.net/event/wildlabs-virtual-meetup-bioacoustics-data-analysis-and-ai
*Thread Reply:* @Sara Beery @Neha Hulkund and I will be there for part of it 🙂
*Thread Reply:* Awesome! Which days will you be there? Are you doing any talks/panels?
We’re (WildMon) part of conservation tech event on the 24th - https://www.cbd.int/side-events/6023
*Thread Reply:* I won't make that one in person unfortunately, but will follow up with a few other events we're taking part in!
*Thread Reply:* That would be awesome thanks! Trying to get a list of nature/conservation-tech-focused events together 🙂
*Thread Reply:* @Carly Batist is the event 6023 going to be broadcasted or can people join remotely?
*Thread Reply:* I believe it’s in-person only unfortunately
*Thread Reply:* interested in any list of conservation tech events compiled and/or people going to be there. i will not personally be there, but likely/hopefully some of my colleagues at CXL will be. gladly pass along info to them.
*Thread Reply:* This isn't related to AI (yet) but I'd be curious to see how solutions from other frameworks (like from the event on the 24th) could be standardized so this Pact could benefit: https://www.linkedin.com/posts/berlin-urban-nature-pact_berlinurbannaturepact-cop16-cities-activity-7254401158918586368-jEpm/
🌍 Ready to Tackle Climate Change? Join the MIT Energy & Climate Hackathon! 🚀
TLDR: Interested in Renewable Energy, Transportation, Future Mobility, Circularity, Climate Change, and tackling real-world problems - all while getting free swag, food, and the chance to win up to $1,000 per person? Apply by October 18, 2024, and participate from November 15-17, 2024, to help create a cleaner future.
Past sponsors include Google, McKinsey Sustainability, Crusoe, Schneider Electric, Fifth Wall, and more.
Here’s what we’re diving into: 🌿 Renewable Energy: Envision a world powered entirely by wind, solar, and green hydrogen—clean energy driving everything we do! 🚀 Future Mobility: Picture electric vehicles, high-speed trains, and even flying taxis making your commute fast, fun, and emissions-free! ♻️ Circularity: Let’s rethink how we use and reuse materials to minimize waste and build a sustainable future.
Join a team of up to 4, collaborate with global competitors, and solve real-world energy challenges proposed by our sponsors. Plus, network with top industry leaders and win up to $1,000 per person!
To participate, fill out our form: https://tinyurl.com/mitec-hack-2024 To learn more about the MITEC Hackathon, check out our website: https://www.mitenergyhack.org/
Please reach out if you have any questions! And email me at jturliuk@mit.edu if you’re interested in your company being a challenge sponsor/partner.
*Thread Reply:* Hey! are recent working grads allowed to participate?
*Thread Reply:* tonight I am taken, but if you want to do smth tomorrow or wednesday, let me know
*Thread Reply:* Maybe we could meet up at the reception tomorrow?
*Thread Reply:* Where could we meet? It will be busy…
*Thread Reply:* What about 1830 at the Google booth?
🦋 Hi All! 👋
We are hosting a workshop on building reproducible workflows for species distribution models (SDMs) using SyncroSim. SyncroSim is a free software for non-commercial use (e.g., academics, non-profits, government). It provides an easy-to-use interface that works with open-source packages, whether through a graphical user interface, command line, R, or Python. SyncroSim simplifies the creation of workflows for SDMs, including models like MaxEnt and random forest. In this workshop, participants will learn the basics of running species distribution models and how to create reproducible workflows for scientific modeling, from start to finish. This workshop is part of the annual Society for Open, Reliable, and Transparent Ecology and Evolutionary biology (SORTEE) conference. Attendance is free for members of SORTEE and membership costs $10-$40 depending on career stage. There are also free waivers for members part of other open science organizations.
📅 Date: Tuesday, October 15, 2024 📍 Location: Online 🕰️ Time: 12:30 pm PT; 3:30 pm ET ✅ Sign up here: https://events.humanitix.com/sortee-conference-2024
New dataset on LILA, containing ~119k images from downward-facing small animal cameras (see the adorable chipmunk at the end of this post):
https://lila.science/datasets/ohio-small-animals/
Thanks to Greg Lipps and Sowbaranika Balasubramaniam from Ohio State for this dataset.
Hopefully this is the first of several datasets from downward-facing cameras, which are not covered in existing public datasets and are not handled well by existing models, including MD.
Hey guys, we need some data scientists to fix the 80% stat. https://news.mongabay.com/2024/09/do-indigenous-peoples-really-conserve-80-of-the-worlds-biodiversity. We’re getting together a scientific workgroup for this that will meet weekly for three months following COP and try do crank out a paper as a coalition the way the
*Thread Reply:* is there a shareable/public version of this scientific workgroup call? i would like to share it with folks not on slack. thanks!
Has anyone read the 2024 WWF report "Living Planet"? Do people understand why the uncertainty in their measurements is increasing with time instead of going down? What's going wrong?
*Thread Reply:* When making predictions about future trends, especially in ecology, the confidence in those projections diminishes because of the unpredictability of various environmental factors (climate change, human impacts, conservation efforts). Thus, future uncertainty grows as a function of both complexity and incomplete data.
*Thread Reply:* Thank you @Aakash Gupta. The top plot does not extrapolate though. The y axis is on a log scale and thus, if the confidence interval is constant in time, the confidence interval will appear to become larger as the index declines. Still, I am surprised that the confidence interval does not shrink in time given all the ongoing work to obtain accurate measurements.
*Thread Reply:* This may also provide some insights: https://communities.springernature.com/posts/the-living-planet-index-is-not-a-reliable-measure-of-population-changes
*Thread Reply:* @Pietro Perona I think it’s because it is indexed to 1970 levels. The error bar size seems like a function of time rather than quality of biodiversity estimates. Since nobody has exact numbers on how many animals there were in 1970 or how many there are today, as we get further from 1970 and more time passes, we can be less and less sure of exactly how far we have fallen the starting point. Basically, in 1971, very little time had passed, so regardless of the quality of biodiversity estimates, the confidence range was small because there just was not enough time between 1970 and 1971 for estimations to get so far away from reflecting what was likely the reality.
*Thread Reply:* We address some of these issues in our recent-ish paper:
*Thread Reply:* Hey Ben! I can share a few couple questions I’d prob ask if estimating to help nail it down more:
• How complex is the ensemble? Do you need GPUs on the windows machines? What kind of software has to be on the machines to make it work (i.e. CUDA, etc) • Is the existing model easily compatible with windows? What format is the model in? • What format and language is the supporting code for the model? How much of it is there? • How technical are the people using the software? Does the update process need to be pretty foolproof or can people download and modify the software themselves?
*Thread Reply:* I don't have a $$$ number, because this probably depends on a lot of factors like where you're located and what you're willing to pay (e.g. are you proposing that a postdoc/grad student does this, or would you contract it out?) Generally when I've submitted academic grants, we've based the amount on length of appointment x salary for position, so a year of postdoc time in the UK might be ~80-100k GBP when you include overheads and other lining of the university's pockets.
Conservatively I'd say it's at least a few months' work. For someone with experience in all the components, it's probably possible to get an MVP out in a matter of weeks. For example if you didn't need a standalone executable, you could use a ready-to-go UI like streamlit/gradio and run it straight from Python/Docker. What will take longer is things like end-user/technical documentation, getting installers working reliably (does it have to be a standalone thing?), etc.
You could find a consultant who could bang this out in much less time, but you'll probably pay for that expertise (think several hundred a day at least).
Realistically, you probably want to have someone around long enough to maintain and update things for at least 6 months while you test it.
*Thread Reply:* Thanks both! Creation of the model is part of the grant so I can only give guesses as to answers, but here's my best intuition currently: • We are aiming to make it lightweight so it can run with no GPU and some patience • It will likely be written in standard libraries like TF, PyTorch. We'll be designing to make it compatible with windows from the off, as this is what are users in the field have. • Assume not technical at all! In my limited experience of this, I'd imagine something like we upload a new checkpoint file/s somewhere and they just replace the old one with this. Let me know if that helps narrow down a range @Nate Harada 🙏
Super useful on the postdoc salary point, fully agree getting something stable is going to be where a lot of the hard yards are
*Thread Reply:* I think that's sensible. Many models can be run fine on CPU if latency isn't an issue. I would suggest decoupling your training library from the "production" model. You can distribute your model in a standard open format like ONNX (e.g. onnx-runtime). You could also consider writing a native application, for example using C++/Qt. That tends to be a lot cleaner than trying to package up a mess of Python dependencies and it will likely run faster. The downside is that finding developers with that kind of experience (in academia), tends to be harder.
If you're also expecting the hire to train the model as well, then budget for at least a year (do you have datasets, etc ready to go?)
Universities tend to have baseline costs for hiring and they're often (much) higher than you expect - e.g. a postdoc in London might get paid 40k, but the University will charge double or more. We've been burned by this in the past (e.g. the HR overhead was so high that we had to scale back the project), so I would highly recommend speaking to someone in HR/grants and getting estimates for staff at different levels of seniority.
*Thread Reply:* Thanks @Josh Veitch-Michaelis! Will certainly be separating the training and deployment aspects, the model will already be developed and we would just be asking the hire to package it up into some kind of desktop app type thing. Good to know on ONNX and C++ etc, thats super helpful. Yeah my PI is experienced in all the HR stuff for grants so we're aware we basically need to double the salary cost for the overheads, we just needed a ballpark figure on this. So we're now budgeting for a 6month post doc and can use these funds to either look for one or for an external contractor, thanks so much for all the tips
*Thread Reply:* Great, best of luck. Feel free to get in touch if you have any other questions!
*Thread Reply:* Sara, are you connected to the team at Planet working on this?
*Thread Reply:* Not directly on this project! But I flagged because it seemed like it might provide opportunities for biodiversity projects to access data.
*Thread Reply:* That's a good point. Though, I'd expect them to be more focused on size considerations. Curious if anyone has any insight and would be interested in the data too!
*Thread Reply:* Well, I wonder if there's any appetite for being able to distinguish between sensitive and non-sensitive species as well, at least from PR perspective
*Thread Reply:* I would certainly hope so 😭 or maybe they don’t want to hit pets or a skunk 🤷♀️
*Thread Reply:* Seems vaguely like (a) yes they do make their own detectors/classifiers for animals but (b) probably extraordinarily painful to collaborate/get access to anything of the kind for an external user of any kind.
*Thread Reply:* I’m not working in this direction but remember Volvo was trying to develop a system to detect animals and the system was confused by kangaroos (news article).
*Thread Reply:* I have a friend who worked in the industry, for lidar data they mostly consider large animals (which crash the car), cats and dogs (potential lawsuits) and birds (flying in front of sensors could lead to problems). Some animal classes were considered but just for evaluation, not for training. Accessing data depends on the company but is often a no-go
*Thread Reply:* Thanks for all the replies, very helpful insights!
For anyone attending COP16 you can now register for our event, which includes a panel "What is the Role of AI in Addressing the Biodiversity Crisis?", and free drinks 🍸
*Thread Reply:* Would love to share code and insights. So far I have been playing the following notebooks: • video predictor • automaticmaskgenerator • ultralytics implementation
*Thread Reply:* I haven't tried SAM2 yet, but I did try out TAM, which was based on SAM1 (https://arxiv.org/abs/2304.11968) A couple notes I had from testing out TAM: • We could only fit ~600 frames at once (a restriction that may exist with SAM2) • SAM supervision works fairly well when things that look identical are separated, but fail by merging them together when they're touching • IR/monochrome imaging isn't well represented in their foundational training data and as such have basic issues for things like shadow influencing borders • Despite those drawbacks, we could pretty reliably get okay tracking with 1 click per object for about 10-15s (of the 20s clip)
*Thread Reply:* Thanks Brian, that sounds interesting!
*Thread Reply:* Hi Filippo, I'm also looking into applying SAM2 for underwater applications! I've been trying to get around having to do point/bounding box prompts so I've been having a look at Grounded SAM2 to do text prompting instead which seems to be giving ok results thus far (I only started playing around with it last week). I'm a bit unclear on whether after some fine tuning I'd be able to automatically generate masks for the specific classes I'm interested in or not however.
*Thread Reply:* Thanks Georgia! I will have a look at Grounding Dino and let’s keep in contact about updates!
*Thread Reply:* can you drop in a screenshot?
*Thread Reply:* and you want to locate them, like with a box or point on the entry.
*Thread Reply:* okay, so depending on how exactly image J was run, you may be able to recover the position of previous annotations over the image. If not, in like an hour on http://labelme.csail.mit.edu/Release3.0/, you probably could label 25 images, and run then through an object detection tool like ours. https://deepforest.readthedocs.io/en/latest/user_guide/11_training.html you can make an discussion on the github repo where you get stuck.
*Thread Reply:* Once you have the manual annotations and/or ML predictions, do you have to worry about double counting burrows across multiple overlapping images? If so, some of the work we've done such as this tool might become useful. You could use this as an additional post-processing step after following the steps Ben described to generate ML predictions on each image.
*Thread Reply:* Thank you so much I will check out these tools! Correct in that we don’t want to double count holes. We were thinking of breaking apart the stitched together orthomosaic.
*Thread Reply:* Cool, this workflow also relies on running structure from motion as a first step, you just generate predictions on the raw images (rather than ortho) and then try to suppress duplicate detections geometrically. The motivation is you may have higher quality detections on the raw images than the orthos, since they don't have stitching artifacts and you get multiple perspectives.
*Thread Reply:* Oh that is very cool
*Thread Reply:* Hi Elke, I've been working on a similar project to detect prairie dog burrows from drone imagery. We trained a model in YOLOv8, I think we are using a similar approach to what David has referenced to merge detections together from different overlapping images. Happy to discuss more with you!
New model is out for wild animal re-id, covering 64 species from mixed data sources.
The is Wild Me's strongest generation of re-id model at use on Wildbook platforms, covering cetaceans, mammals, carnivores and more. The model shows state-of-the-art generalization ability for search queries on Wildbook platforms. For this release, the model is stripped down and made easy to run and experiment with.
Model weights and List of species included Ready-to-use scripts
Hopefuly this helps not only ones that have or have not worked with MiewID previously. The knowledge transfer lowers the barrier and opens up the possibility to adapt deep learning models to datasets previously too small to train.
*Thread Reply:* From-scratch vs leveraging model weights to adapt to new datasets.
*Thread Reply:* A paper with detailed zero-shot and fine-tuning studies will be available next week. Model also transfers well to 'similar' species in a 0-shot manner, for example, the dataset includes lots of dorsal fin samples, so the model just generally does well at them, regardless of species. Moreover, the 64 species can easily be fine-tuned for a new domains to make use of knowledge transfer for a general re-id task. The positive effect varies on per-species basis from limited to dramatic. The knowledge transfer could especially be handy for those working with just small amounts of labelled data.
*Thread Reply:* Awesome work! Can you please also post the mentioned publication here as well? I am very interested in reading it!
Check out our robust and efficient feature extractor (Pytorch) of bird sounds, with only ~20M parameters, trained on the Xeno-Canto dataset, covering over 10,000 bird species, ideal for bioacoustic projects. Explore it here: https://huggingface.co/ilyassmoummad/ProtoCLR
*Thread Reply:* Classification accuracy of ProtoCLR compared to various models on one-shot and five-shot bird sound classification tasks
*Thread Reply:* am I wrong in thinking that's what this slack channel is for? or https://wildlabs.net/ ? I am a funder who is involved with various conservation efforts and I try to support pre-existing communities rather than create new ones... Could you give a little more information on your goals for this group?
*Thread Reply:* I totally understand where you're coming from! This is a group that is all facets of conservation, not just AI or technology. This means natural solutions from ocean conservation to mangrove restoration. The goal is to connect people from all facets of conservation - not just AI nor technology.
*Thread Reply:* Hey cool tool! When I go to the link for a website i get 'There's been a glitch' because it takes me to slack.com/yourwebsite, your actual website works!
*Thread Reply:* Oh you are right, thank you for the notice! 💡
It was an extra "/" in the link, whoops. 😇
